{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path = ['/Users/sebastian/github/mlxtend/'] + sys.path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sebastian Raschka \n",
      "\n",
      "CPython 3.4.3\n",
      "IPython 3.0.0\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -a 'Sebastian Raschka' -v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Artificial Neurons and Single-Layer Neural Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## - How Machine Learning Algorithms Work Part 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**This article offers a brief glimpse of the history and basic concepts of machine learning. We will take a look at the first algorithmically described neural network and the gradient descent algorithm in context of adaptive linear neurons, which will not only introduce the principles of machine learning but also serve as the basis for modern multilayer neural networks in future articles.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [Introduction](#Introduction)\n",
    "- [Artificial Neurons and the McCulloch-Pitts Model](#Artificial-Neurons-and-the-McCulloch-Pitts-Model)\n",
    "- [Frank Rosenblatt's Perceptron](#Frank-Rosenblatt's-Perceptron)\n",
    "    - [The Unit Step Function](#The-Unit-Step-Function)\n",
    "    - [The Perceptron Learning Rule](#The-Perceptron-Learning-Rule)\n",
    "    - [Implementing the Perceptron Rule in Python](#Implementing-the-Perceptron-Rule-in-Python)\n",
    "    - [Problems with Perceptrons](#Problems-with-Perceptrons)\n",
    "- [Adaptive Linear Neurons and the Delta Rule](#Adaptive-Linear-Neurons-and-the-Delta-Rule)\n",
    "    - [Gradient Descent](#Gradient-Descent)\n",
    "    - [The Gradient Descent Rule in Action](#The-Gradient-Descent-Rule-in-Action)\n",
    "    - [Online Learning via Stochastic Gradient Descent](#Online-Learning-via-Stochastic-Gradient-Descent)\n",
    "- [What's Next?](#What's-Next?)\n",
    "- [References](#References)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Machine learning is one of the hottest and most exciting fields in the modern age of technology. Thanks to machine learning, we enjoy robust email spam filters, convenient text and voice recognition, reliable web search engines, challenging chess players, and, hopefully soon, safe and efficient self-driving cars. \n",
    "\n",
    "Without any doubt, machine learning has become a big and popular field, and sometimes it may be challenging to see the (random) forest for the (decision) trees. Thus, I thought that it might be worthwhile to explore different machine learning algorithms in more detail by not only discussing the theory but also by implementing them step by step.\n",
    "\n",
    "To briefly summarize what machine learning is all about: \"[Machine learning is the] field of study that gives computers the ability to learn without being explicitly programmed\" (Arthur Samuel, 1959). Machine learning is about the development and use of algorithms that can recognize patterns in data in order to make decisions based on statistics, probability theory, combinatorics, and optimization.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first article in this series will introduce perceptrons and the adaline (ADAptive LINear NEuron), which fall into the category of single-layer neural networks. The perceptron is not only the first algorithmically described learning algorithm [[1](#References)], but it is also very intuitive, easy to implement, and a good entry point to the (re-discovered) modern state-of-the-art machine learning algorithms: Artificial neural networks (or \"deep learning\" if you like). As we will see later, the adaline is a consequent improvement of the perceptron algorithm and offers a good opportunity to learn about a popular optimization algorithm in machine learning: gradient descent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Artificial Neurons and the McCulloch-Pitts Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The initial idea of the perceptron dates back to the work of Warren McCulloch and Walter Pitts in 1943 [[2](#References)], who drew an analogy between biological neurons and simple logic gates with binary outputs. In more intuitive terms, neurons can be understood as the subunits of a neural network in a biological brain. Here, the signals of variable magnitudes arrive at the dendrites. Those input signals are then accumulated in the cell body of the neuron, and if the accumulated signal exceeds a certain threshold, a output signal is generated that which will be passed on by the axon.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_neuron.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Frank Rosenblatt's Perceptron"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To continue with the story, a few years after McCulloch and Walter Pitt, Frank Rosenblatt published the first concept of the Perceptron learning rule [[1](#References)]. The main idea was to define an algorithm in order to learn the values of the weights $w$ that are then multiplied with the input features in order to make a decision whether a neuron fires or not. In context of pattern classification, such an algorithm could be useful to determine if a sample belongs to one class or the other. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_binary.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To put the perceptron algorithm into the broader context of machine learning: The perceptron belongs to the category of supervised learning algorithms, single-layer binary linear classifiers to be more specific. In brief, the task is to predict to which of two possible categories a certain data point belongs based on a set of input variables. In this article, I don't want to discuss the concept of predictive modeling and classification in too much detail, but if you prefer more background information, please see my previous article \"[Introduction to supervised learning](http://sebastianraschka.com/Articles/2014_intro_supervised_learning.html)\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Unit Step Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we dive deeper into the algorithm(s) for learning the weights of the artificial neuron, let us take a brief look at the basic notation. In the following sections, we will label the *positive* and *negative* class in our binary classification setting as \"1\" and \"-1\", respectively. Next, we define an activation function $g(\\mathbf{z})$ that takes a linear combination of the input values $\\mathbf{x}$ and weights $\\mathbf{w}$ as input ($\\mathbf{z} = w_1x_{1} + \\dots + w_mx_{m}$), and if $g(\\mathbf{z})$ is greater than a defined threshold $\\theta$ we predict 1 and -1 otherwise; in this case, this activation function $g$ is an alternative form of a simple \"unit step function,\" which is sometimes also called \"Heaviside step function.\" \n",
    "\n",
    "*(Please note that the *unit step* is classically defined as being equal to 0 if $ z < 0$ and 1 for $z \\ge 0$; nonetheless, we will refer to the following piece-wise linear function with -1 if $z < \\theta$ and 1 for $z \\ge \\theta$ as *unit step function* for simplicity).*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    " g(\\mathbf{z}) =\\begin{cases}\n",
    "    1 & \\text{if $\\mathbf{z} \\ge \\theta$}\\\\\n",
    "    -1 & \\text{otherwise}.\n",
    "  \\end{cases}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where\n",
    "\n",
    "$$\\mathbf{z} =  w_1x_{1} + \\dots + w_mx_{m} = \\sum_{j=1}^{m} x_{j}w_{j} \\\\ = \\mathbf{w}^T\\mathbf{x}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\mathbf{w}$ is the feature vector, and $\\mathbf{x}$ is an $m$-dimensional sample from the training dataset:\n",
    "\n",
    "$$ \n",
    "\\mathbf{w} = \\begin{bmatrix}\n",
    "    w_{1}  \\\\\n",
    "    \\vdots \\\\\n",
    "    w_{m}\n",
    "\\end{bmatrix}\n",
    "\\quad  \\mathbf{x} = \\begin{bmatrix}\n",
    "    x_{1}  \\\\\n",
    "    \\vdots \\\\\n",
    "    x_{m}\n",
    "\\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_unit_step.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to simplify the notation, we bring $\\theta$ to the left side of the equation and define $w_0 = -\\theta  \\text{ and } x_0=1$ \n",
    "\n",
    "so that \n",
    "\n",
    "$$\\begin{equation}\n",
    " g({\\mathbf{z}}) =\\begin{cases}\n",
    "    1 & \\text{if $\\mathbf{z} \\ge 0$}\\\\\n",
    "    -1 & \\text{otherwise}.\n",
    "  \\end{cases}\n",
    "\\end{equation}$$\n",
    "\n",
    "and\n",
    "\n",
    "\n",
    "$$\\mathbf{z} = w_0x_{0} + w_1x_{1} + \\dots + w_mx_{m} = \\sum_{j=0}^{m} x_{j}w_{j} \\\\ = \\mathbf{w}^T\\mathbf{x}.$$\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Perceptron Learning Rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It might sound like extreme case of a reductionist approach, but the idea behind this \"thresholded\" perceptron was to mimic how a single neuron in the brain works: It either \"fires\" or not. To summarize the main points from the previous section: A perceptron receives multiple input signals, and if the sum of the input signals exceed a certain threshold it either returns a signal or  remains \"silent\" otherwise. What made this a \"machine learning\" algorithm was Frank Rosenblatt's idea of the perceptron learning rule: The perceptron algorithm is about learning the weights for the input signals in order to draw linear decision boundary that allows us to discriminate between the two linearly separable classes +1 and -1.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_schematic.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rosenblatt's initial perceptron rule is fairly simple and can be summarized by the following steps: \n",
    "\n",
    "1. Initialize the weights to 0 or small random numbers.\n",
    "2. For each training sample $\\mathbf{x^{(i)}}$:\n",
    "    2. Calculate the *output* value.\n",
    "    2. Update the weights."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output value is the class label predicted by the unit step function that we defined earlier (output $=g(\\mathbf{z})$) and the weight update can be written more formally as  $w_j := w_j + \\Delta w_j$.\n",
    "\n",
    "The value for updating the weights at each increment is calculated by the learning rule\n",
    "\n",
    "$\\Delta w_j = \\eta \\; (\\text{target}^{(i)} - \\text{output}^{(i)})\\;x^{(i)}_{j}$\n",
    "\n",
    "where $\\eta$ is the learning rate (a constant between 0.0 and 1.0), \"target\" is the true class label, and the \"output\" is the predicted class label."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to note that all weights in the weight vector are being updated simultaneously. Concretely, for a 2-dimensional dataset, we would write the update as:\n",
    "\n",
    "$\\Delta w_0 = \\eta(\\text{target}^{(i)} - \\text{output}^{(i)})$  \n",
    "$\\Delta w_1 = \\eta(\\text{target}^{(i)} - \\text{output}^{(i)})\\;x^{(i)}_{1}$  \n",
    "$\\Delta w_2 = \\eta(\\text{target}^{(i)} - \\text{output}^{(i)})\\;x^{(i)}_{2}$  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we implement the perceptron rule in Python, let us make a simple thought experiment to illustrate how beautifully simple this learning rule really is. In the two scenarios where the perceptron predicts the class label correctly, the weights remain unchanged:\n",
    "\n",
    "- $\\Delta w_j = \\eta(-1^{(i)} - -1^{(i)})\\;x^{(i)}_{j} = 0$ \n",
    "- $\\Delta w_j = \\eta(1^{(i)} - 1^{(i)})\\;x^{(i)}_{j} = 0$ \n",
    "\n",
    "However, in case of a wrong prediction, the weights are being \"pushed\" towards the direction of the positive or negative target class, respectively:\n",
    "\n",
    "- $\\Delta w_j = \\eta(1^{(i)} - -1^{(i)})\\;x^{(i)}_{j} = \\eta(2)\\;x^{(i)}_{j}$ \n",
    "- $\\Delta w_j = \\eta(-1^{(i)} - 1^{(i)})\\;x^{(i)}_{j} = \\eta(-2)\\;x^{(i)}_{j}$ \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to note that the convergence of the perceptron is only guaranteed if the two classes are linearly separable. If the two classes can't be separated by a linear decision boundary, we can set a maximum number of passes over the training dataset (\"epochs\") and/or a threshold for the number of tolerated misclassifications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementing the Perceptron Rule in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, we will implement the simple perceptron learning rule in Python to classify flowers in the Iris dataset.\n",
    "Please note that I omitted some \"safety checks\" for clarity, for a more \"robust\" version please see the following [code on GitHub](https://github.com/rasbt/mlxtend/blob/master/mlxtend/classifier/perceptron.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class Perceptron(object):\n",
    "    \n",
    "    def __init__(self, eta=0.01, epochs=50):\n",
    "        self.eta = eta\n",
    "        self.epochs = epochs\n",
    "\n",
    "    def train(self, X, y):\n",
    "\n",
    "        self.w_ = np.zeros(1 + X.shape[1])\n",
    "        self.errors_ = []\n",
    "\n",
    "        for _ in range(self.epochs):\n",
    "            errors = 0\n",
    "            for xi, target in zip(X, y):\n",
    "                update = self.eta * (target - self.predict(xi))\n",
    "                self.w_[1:] +=  update * xi\n",
    "                self.w_[0] +=  update\n",
    "                errors += int(update != 0.0)\n",
    "            self.errors_.append(errors)\n",
    "        return self\n",
    "\n",
    "    def net_input(self, X):\n",
    "        return np.dot(X, self.w_[1:]) + self.w_[0]\n",
    "\n",
    "    def predict(self, X):\n",
    "        return np.where(self.net_input(X) >= 0.0, 1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the following example, we will load the Iris data set from the [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml/) and only focus on the two flower species *Setosa* and *Versicolor*. Furthermore, we will only use the two features *sepal length* and *petal length* for visualization purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)\n",
    "\n",
    "# setosa and versicolor\n",
    "y = df.iloc[0:100, 4].values\n",
    "y = np.where(y == 'Iris-setosa', -1, 1)\n",
    "\n",
    "# sepal length and petal length\n",
    "X = df.iloc[0:100, [0,2]].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weights: [-0.4  -0.68  1.82]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEZCAYAAABoy7XpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcnWV5//HPN5mEBBhIIIGwBANKgiCyyiKmULRICsGi\nqMW2SnHFWoOF/IzIr3WpthbBmv5UUJEKpW4sVRZFYn6gSF+sSUGCCbKZABPZEhIgyyRX/3ieM3ly\nmLPMOfPMOec53/frNa85z36fk8k191zPfV+PIgIzMyuGUa1ugJmZDR8HdTOzAnFQNzMrEAd1M7MC\ncVA3MysQB3UzswJxUDczKxAHdWtrkh6T9JKkNZL6JF0mabtWtwtA0mckXdHqdphlOahbuwvg5Ijo\nBQ4FDgfOr/dgpfJqXLte27qXg7p1jIh4EvgZ8DpJR0m6XdLzkhZLOra0n6RbJP2jpF8DLwJ7SzpA\n0s2Snk17/J9K9x0laZ6k30l6RtIPJE1Mt02TtFnSByU9IelJSeek204EPgW8O/0rYlGVa79R0l2S\nVkm6U9LRZW39nKTbJL0g6SZJO4/MJ2pF5KBunUAAkqYCs4CngOuBz0XEROBc4OqyYPiXwAeA7YGn\ngQXAjcBuwGuAX6T7/S1wCvBH6bbnga+VXf+49JgTgE9KenNE/Az4IvD9iOiNiEMqXPtF4AbgX4Gd\ngIuAG0q/OFKnA2cAuwBj0/dj1hAHdWt3Av5L0vPAr4BbgBXAjWlgJSIWAHcDJ6XHBPDvEfFgRGwG\nTgaejIivRMSGiFgbEXem+34YOD8inoyIjcBngdMkZf9vfDYiXo6I3wCXkQThUtvK0yvl1z4BWBoR\nV0bE5oj4PvBbkl8kpf0vi4jfRcQ64IfAwU19YtbVelrdALMaAnhbRCwsrZD0deCdkmZn9usBFmaW\nl2deTwUeqXD+acC1kjZn1vUDu1Y41++BA2u0Obv/7ukxWY+n60v6Mq9fJunhmzXEPXXrRL8HroiI\niZmv3oj4l8w+Ubb/PlXOdWLZubaNiKcy++xV9vqJQa6RlV3/BPCqsu2vypzDbFg5qFsn+g9gtqQT\nJI2WNE7ScZL2yOyTTYtcD+wmaY6kbST1Sjoi3XYx8EVJewFImizpFLZ2vqTxkg4gyX3/IF3fB0wb\nZIRLdvlGYLqk0yX1SHo3sF/apsH2N2uKg7p1nIhYAbwNOA/4A0lv+xy2Do6R2X8t8CfAbJKbrMtI\nbn4CfBX4CfBzSS8A/w0cwdZuBX5HcrP1gjSHD/Cj9Puzku6ucO3nSHL65wDPkNwEPTld/4r909d+\nyIE1TH5IhtngJE0jycX3pDc9zdqee+pmZgXioG5Wnf+UtY6Sa1CXNEHSVZIelLRE0lF5Xs9sOEXE\nYxEx2qkX6yR5j1P/KskkkdMk9QBtUYjJzKyocrtRKmlHYFFEVBofjCT/aWtm1oCIGHQobJ499b2B\npyVdBhwE3APMiYiXypr24RzbMIJmzYafXtfqVrQnfzbV+fOpzJ/N4HRJpS155tR7SEqlfj0iDiUp\nbDQvx+uZmXW9PHvqK4AVEXFXunwVgwb1WZn6HccvhbnLcmyTmVkHumA6LJxRz565BfWI6JO0XNL0\niFgGvAV44JV7FuVPq+OXtroF7cufTXX+fCrzZ5OYu2zrDq9OrrRnrjNKJR0EfJukRvTDwF9HxOrM\n9ihOTt3MbKToklbcKCUi/gd4Q57XMDNLPDpuypQvHTthwpopUmcXSYsgVq3q7evr++StsPe6oRzr\neupmVghTpnzp2Pe//8RdjjzypOd6esa0ujlN6e/fyB133LDLpZd+6di+votvGsqxLhNgZoUwYcKa\nKUceedKqTg/oAD09YzjyyJNWTZiwZspQj3VQN7NCkFARAnpJT88YGkkjOaibmRWIg7qZWYH4RqmZ\n2QhasuTXvd/85if2X7Fi6Y4vvbRm7I03br6+9lH1c0/dzGwEjRkzbvMxx7zjyQ9+8KL/yeP87qmb\nWVc754P7zex/8fmx2XU9203ccOG3fvurPK63776Hvbjvvoe9+PDDi7bN4/wO6mZWaOVBuzxg97/4\n/Nh7t5uwIXvMoWVBvpM4qJtZoZUH7UYCdq1fDO3EQd3MrIZmfjFce+1X9vjudz/9eoB99jn42Ysu\nuv3OPNpY4qBuZl2tZ7uJG8qDdM92EzdU2n+oTj31E0+ceuonnhiu89XioG5mhVYetMsDdivSKC+/\nvHbUxo3rRwGsW/fSKIBx47YdlgecO6ibWaENR9Cu9YthKB5//Dfjzzrr9W8GkMTb3779n06cuOtL\nV1751MJm2wkO6mZmNQ1nb/5Vr3rdy8M94SjLk4/MzArEQd3MrEAc1M3MCsRB3cysQBzUzcwKxEHd\nzKxAHNTNzArEQd3MrEAc1M3MCsQzSs3McnTFFf932i9/+cOpK1c+2nvooW994jOfuS6XJx6VuKdu\nZl1t+XLG9fdvWX78ccYP5/knTZq67rTT5i476qi3LR/O81bioG7WGr3A1Mzy1HSdDbO+PrYpvd6w\nAT37LGOy26++mmlf/zoz+vvh5pvZ5atf5cC1axk9XNefNetDfW996wdWbr/9TsNWzrcaB3Wz1pgA\nnAbslX6dlq6zYfTUU2zzT//EIYsXs+OGDWj+fF577bXsld3nIx9h2dq1jDnrLGbecAN7z5nD/dtv\nz6bS9t/+lu2vvpo9S8s/+xlT7r67kX+raOat1C33nLqkx4AXgE3Axog4Iu9rmnWA5cDVwF+my/+R\nrrNhtNturD/jDB789rfZH2DaNFa/9708kt1n3Dg2H3QQzzz6KBN7e9mwxx68nN0+aRIb7rqLXfv7\n0Q47sHHBAvY6+2zuG3pr1MxbqdtI9NQDOC4iDnFANyuctk8jHXAALwCsX0/Pccfx5NixW3eZb76Z\nXRYs4FVz53Lv+PFsLKViSiZNYsO553L/ggVMu+Ya9j37bO6bOpV1Q2/JyPTURyr9MjK/osw6x1Tg\nHSQ99P9IX0+tekR7aus0UinlMm0aqz/0Ie6//HL2W7yYHbP7rF7N2LPP5r7p03nx7/6OJTvuyIYN\nG7bOqd91FzuNGkWMGkXcfjuTG2vNyITBkRjSGMACSZuASyLiWyNwTbN2twq4ii0pl6vSdZ2mrdNI\nL7zAmAkTWP/e9/LI2LFETw8PPvYY2x98MKtL+5x2GitKr8eNY/P73sej2XPcdRcTFyxgr/PP5+7x\n49n85S9z4KRJrH/zm/lDPW3o79/Ixo3rR23e3K/Nmzdp3bqXRvX0jNnc0zOm9sENGImgfkxEPCVp\nMnCzpN9GROYpIrNmb3l9/FKYu2wE2mTWamvSr5K2CYRFMmkSGz7wAR4uLR9yCKsPOWRLQK/HQQex\naq+9WLzrrmwAmDuX+8aP33IjtZaLL/749BtvvGR6afntb99+z5NPPmvpRz/6tYfqb8UF02HhjHr2\nzD2oR8RT6fenJV0LHAFkgvpPr8u7DWZdopck9VH6BTGVpPe/puIRzcumkUhfZ/8CqaQVbW3I2LFE\nKaAD7LwzG4dy/Mc+9o1lH/vYN5rsrM5dtnWHVydX2jPXnLqkbSX1pq+3A04A7s/zmmZdrBX57VIa\n6ffpV71ppLbOxXeyvHvquwLXSipd68qI+HnO1zTrVq3IbzeaRmrrXHwnyzWoR8SjwMF5XsPMzLbw\njFKzfLRi/Ha7DZOs9hm0W1sLw0HdLB+dlN/OS7XPYNjbGkH09w/pHmZb6+/fSMTQZyy59K5ZPjop\nv52Xap/BsLd11arevjvuuGGXI488aVVeY8BHSn//Ru6444YJq1b19g31WAd1s/azG7AHcHe6fDjw\nBPBUjeNaMUwwj2s2dM6+vk/eeumlXzr26qt/NEXq7FnsEcSqVb19fX2fvHWoxzqom+Wj0fHbkAT0\nv2BLevR04EpqB/VSuuPqsmvmGdSrXbPRz6DB97H3ur6+i2/qG3LftlgUMTJFZga9uBQQH25ZA8zy\n02wP9gjgr9LXVwB31nncXmyd7vh9ncc1o9I1m/kMWvE+OoguiYhB/xpxT90sH+2W324FfwYt4KBu\n1n5mkqQfrkiX/wLYhq3Kawwqryn71bY3k2Ya7vdheEijWTtaCzwL9KVfz6braslryv6IDk3M6Zxd\nwzl1s/Y00jnlWtdzjrutVM6pu6duZlYgDupWJI1Oza92XF7T/XcjGX9ecni6rnSNSlPo83iPtabs\ne0p/B3FQtyJpdGp+tePymu5fGot+RPr1F+k6qJ5TzuM91sphO8fdQZxTt6JpNPdb7bi88skjPRbd\nefHCcE7drO2fej8M8nqPrUhPWYMc1K1IquV+q6Ufqh2XVz75cJLp/1ekX6ezdY69kjzeYy2tSE9Z\ng5x+sSKpNYmmkenseRXJyqtoVx5T9qudt9Y2y4XLBFh3aHRaerXj8prq/hRbB/C7K+1YJo/3aAXi\n9It1i6IMy2t0aGIzue9WpKesQU6/WLdoRa3xPExl8LK0y6ldo6XScbW0Ij1lVVVOvziom3UeD2ns\neh7SaGbWFRzUzarnm6tN52/0nM1oNIft3HeXcFA3qz7Wutp0/kbP2YxGp+x7qn+XcE7dLFEt39xJ\nj5azruCculk1vcCumeVd2Xoa/M6ZbTvjKfLWxipOPpJ0XR3HPxcR7xvG9pi1wgzgPcD30uX3AFeS\nTAh6I3AK8JN02ynAZuAmqj/13o9ks5aoNqN0P+ADwGBd/EjXfy2PRpmNsKXAfwJ/ki7/Z7oO4D6S\nn/VD0+Ufp+sgCdBXs3WKpRS0Szns0rJz2DYiqgX18yPi1moHS/pcrQtIGk3S41kREbOH2D6zkbAG\nWJlZXsmWyTNPAWPYEtSXULs+S+mcnpZvI65iTj0iflDr4Hr2AeaQ/Edo3R1Zs8afNFRt234kqZrS\ntvek6+rhcraWi5o3SiW9QdK1khZJuj/9uq/WcemxewJ/CnybwdM4ZiOl0ScNVds2WEel3s6Ly9la\nLuqp0nglcC7wG5IbREPxFWAusMMQjzMbbneTdGKyQxNLlREbrdJYysU3MmyxWj6+2jazquoJ6k9H\nxE9q77Y1SScDf4iIRZKOq7znrEye/filMHfZUK9lZlZsF0yHhTPq2bPm5CNJJwDvBhYAG9LVERHX\n1DjuiyS9on5gHElv/eqIeG9mH08+snJ5Vf07nCTlUhq2eDpbhi02es0ZwKkkvXVIcurXsmXkTDXV\nqiY2U1HRukJzD8l4H8kPbw9bp1+qBvWIOA84D0DSscC52YBuVkG1sd/NeIItQRySn+UnmrzmYP+p\n6r13VG3Io4dDWsPq6akvBfaLJuoJpEH9nIg4pWy9e+o2mFZMr3c5W+sgzZUJuB3Yv5nLR8St5QHd\nbIR5mKB1hXrSL0cDiyU9CqxP10VEvD6/ZlkXy2t6fR5T+l0KwNpOPemXaYOtj4jHmr640y/2Snk+\nHq1SqqTRa/pRbtYizd0onQIsiYgXACTtALwWeGzY2me2RSum1zd6TZcCsLZTT079YmBtZvnFdJ1Z\nJ2lmSr9Zx6irnnpEbM683gSMzq1FZvloZkq/WceoJ/3yqKSPA98gGYN7FvBIrq0yG37NTOk36xj1\n9NQ/AhxDMlFjBXAU8KE8G2VmZo2p2VOPiJUkZQLMOlm14YcexWKFUbGnLqlmb7yefczaRLUSui51\na4VRrac+T9IzVH+c3dnAN/NomNkwqzb80KVurTCqBfVfArUeP/fzYWyLFUtRUhpFeR/WJSoG9Yg4\nYwTbYcWTV7XFPFTLt3fS+zCrXSYg14u7TEDRdUoFw1q98U55H9Y1misTYFZ0nu5vheGgbnkpSgXD\norwP6xL1VGkcR/KDPI0tvwQiIj7X9MWdfimyotxgLMr7sEJpLv3yY5If4nuAdcPZLCu0oqQ0ivI+\nrEvUE9T3iIi35t4SMzNrWl2Ps5PkpxzZYKo9Is6PjzNrgWplAu6XdD/wJuAeSctK6yTdN3JNtDZW\nbXq9p96btUC19EtpNmmpJECW61AbVJ9e76n3Zi1QbUbpYwCSroiIv8puk3QF8FeDHWdmZq1Tz43S\n12UXJPUAh+XTHOsw1cZwe3y3WQtUDOqSzgM+BYyXlB3StRFXZrREqZxtKVBny9lW22ZmjTuz2sZ6\nJh/9c0TMG9YmbTm3Jx+ZmdXQy+ov7MiqgYEGK5jW08zkox9JOrRs3Wrg8Yjob6KdZmaWyPa+d9mT\nx6eW77CcafcA3wEQXFLpRPUE9a+R5NBLwxgPBB4AdpR0VkTcVG+rzcws0cvqL4xj3bhtWDcO4Eju\nHJixfxXvWg78cyPnrSeoPwm8PyIeAJC0P/B54P8A1wAVg3paN+ZWYBtgLPDjiPhUIw01M+tg86r0\nviHtgQ+HeoL6jFJAB4iIJZL2i4iHk5x4ZRGxTtIfR8RL6aiZ2yS9KSJua7bhZmZt6EyAPXn8FSME\nlzOt4d73UNQT1B+Q9A3g+ySTkN4FLJG0DclImKoi4qX05VhgNPBcg2214nDlQyuCeZNZObm0kE2j\nzOS2VXOY/+lWNKqeoH4G8FGSh0wD/Bo4lySgH1/rYEmjgHuBVwPfiIglDbXUisSPiLNOc2aF3vc9\nmcWHgF+NXJMGN2KPs5O0I0n+fV5E3JKu85DG7uVHxFk7Ghi+3cvq3uwwwpFKn9RD0Hg9dUlvAv6B\nVz4kY5+hNCIiVku6ATgcuGXLllmzt7w+finMXTaU85qZNejM3VlxcGlhFJtGA5zLRQOT5OYw//u0\nQe/7Api+EGbUs289k4+WkqRe7gU2ldZHxDM1Ty5NAvojYpWk8SQ99c9GxC/S7e6pd6epDJ5+cQkB\ny8PM0ovJrPyzUu4bBnrfv87s2/IAXo+meurAqoj4aYPX3g34bppXHwVcUQro1tVcQsDyMq+X1QN1\n+0vpk0k8M9AhXcThbdH7zktdZQJIRq1cA6wvrY+Ie5u+uHvqZtakyay8cJDed8kfGMYx4O2i2Z76\nUST10w8vW//HzTbMzKwOM4F9AXpZ/erszUtIeuFF730PRc2gHhHHjUA7zMwAzuxl9atLC6UAXjaF\nfqAGir1SPaNfpgBfIHkA9YlpmYCjI+LS3FtnZoVXXoHwXC5aNYf5D2d2cQAfgnrSL/8OXAaUZkc9\nBPwQcFA3s3oMqQKhNaeeoD4pIn4gaR5ARGyU5JK7ZlZRXhUIrbZ6gvpaSTuXFiQdRVJP3cwMRrAC\nodVWT1A/B7gO2EfS7cBkkokjZtZd5gE4fdLe6hn9co+kY9kyRXVpRNSszmhmnauX1V/oZc3AJJ7s\nFPo5zHf6pI1Ve/D0O0jGpyvzHWC6JCLimhFon5nlayZwTJXe90OZVR4H3gGq9dRnkwTzShzUzTrL\nwBT60k3M0vT5RRzu3ndBVAzqEXHGCLbDzIbXvN1ZsXspbVKSmUK/hqSQlXvfBVPPjVIza1+1KhC6\n991lHNTNOkfVCoSv5pGNV/Eu10Dpcg7qZm2sRgXCh/EwQitT7+iXcuHRL2bDYmAKfZUKhH8z8s2y\nTuXRL2YjbDIrL4Stnz5f2ub0iTXLo1/MclRegRAGet+L00WnT2xY1ZVTl3QysD8wkNuLiM/l1Siz\nDlRtCr173zZi6qmnfgkwHjge+BbwTuCOnNtl1rYms/LCMWwcU1oexabRR3Lnupnctm4O810DxVqq\nnp76GyPiQEn3RcRnJV0I/Czvhpm1gZm9rD6xQvrkR5lV7oVb26gnqL+cfn9J0h7As8CU/Jpk1hLz\nSi9KMzEzU+jd+7aOUU9Qv17SROACoFQf+Vv5NcksX4NVIEx730+mq57EMzGtQymi2qhFkDQuItaV\nXpPcLF1XWtfUxaWA+HCz5zGrYGAK/e6seGe2DkpZBUKnT6yjCC6JiMHmENXVU78dOBQgDeTrJN1b\nWmfWDnpZ/YXS6/IKhDAwhf7jrWmd2cipNqN0N2B3YFtJh7KlrvoOwLYj0zyzimYO0vt2BULretV6\n6icAZwB7ABdm1q8BzsuxTWYlA1PoJ7PygGwNFEhmYrr3bba1ajNKvwt8V9JpEXHVCLbJulDp6fOl\n5QpT6F0DxayGenLqt0m6FNgjIk6UtD9wdERcmnPbrOB2Z8X8QW5eljyE0ydmQ1ZPUP934DLg0+ny\nQ8APgZpBXdJU4HJgF5J8/DcjYn5DLbVONDD2ezIrJ5enT1yB0Gz41RPUJ0XEDyTNA4iIjZL66zz/\nRuATEbFY0vbAPZJujogHG22wtbfSFPrs0+fTTevmMP+/cO/bLFf1BPW1knYuLUg6Clhdz8kjog/o\nS1+vlfQgyYgaB/UCKH+AAwzcvCxNoXcANxth9QT1c4DrgH0k3Q5MBk4b6oUkTQMOwcXAOs1M4Jhe\nVvdWqIHiCoRmbaRmUI+IeyT9ETCDZKz60ojYOJSLpKmXq4A5EbF2662zZm95ffxSmLtsKOe24eUK\nhGbt5wKYvjCJwTXVUyZgPPBR4E0kNzt/BXyj3jIBksYA1wM/jYh/LdvmMgGtMRPAFQjNOlOzZQIu\nB14A5ifn4j3AFSR11atfWBLJKJkl5QHdRk52Cn0va3qHWoFwBpy3CQYKYI2GNUvhi820KY9zmll9\nQf2AiNg/s7xQ0pI6z38M8JfAfZIWpes+FRGux56TKo9PK1UgXMMQKxBugt7fJccB8JpMMG5UHuc0\ns/qC+r2Sjo6I/4aB0S/31DgGgIi4DRjVRPussoEp9Luz4uCySTy+eWnWpeoJ6ocDv5a0nCSnvhew\nVNL9QETE6/NsoA0+hX4Sz2x6NY+Ubli7AqGZAfUF9RNzb4WVm7knj/95dkXZFPoRHYEyGta8piz/\n3Y7nNLM6Rr/kevHuHv1yJkn5hIHHp2U3nstFq+Yw/9ODHmlmXa3Z0S82PGZOZuWfwZb0yRlcXuqd\nrnEAN7Ph4KCek/IKhDDQ+34YVyDMbUjjZPi3nszPdT/0Pw1/28w5PfzSOomDenPOBHYZrAKhH+BQ\nXV5DGnug5ykYKDi32zD8jHv4pXUSB/X6zdydFQMTrsqqELoCoZm1BQf1V5oJMJmVf1ah9/1fmVUO\n4mbWVro9qM/sZfXAkM3STMzMFPqueYBDtbxxo3nqiXDxNklpCQDWQzwPHwFYDdvvBgMzX/szKZNm\n9EP/bmVtbfacHn5pnaSrgvpgNy/T3vfT6eKQp9AXRbW8caN56m1AfcmENQCmZAL8jrA2jzx1szdF\nB+ObotZJihjUZwL7lhb25PHDSq/9+DQzK7qOD+rZKfSDPYF+Jrd5Eo+ZdY1ODOpnZnvfafrkgcx2\nP8RhELXGWlfLG1fLU1fLm69n65TL+kx7noWdpsBOg22rlcPPI/9fjcepWydpx6BeMX1Sspxpy+nS\n3Hejao21rhakqgXFannzneG5StfcBqh0XK0cfh75/2o8Tt06ScuD+mRWXlh6XXoKvdMnZmaNaXlQ\n34MVYxZx+OLMKqdPqmg0FfAsTNwjk+54KdNLrqVWSmNzpped9RxMPChzzefKrrmpwnGbYdQSGJNZ\nHnS/oXIaxbpBy4P6Ig73VPohaDQVsA3od5nlvYcQKKulNNYDu2f2zebGx4C+lwnkf1yWX9+jwnEb\nIU7P7Lux7JdBo/n/Rj87j1O3TtLyoG4jQ7B5fCbAaZj+7avlzUfB5v0z1xyVueY42FzpF8VO8Pz/\n5JD/b5R789ZJ/Kg5M7MCcU+9wzwLE8uGAg6kJqrljGtNn2/02GqpiXWgKTB2sLY2es5aqr2PvEoT\nmLUTB/UOMw7iKSg9m3SrtEW1nHGttESjx1ZLTewMzw/3OWup9j7yKk1g1k6cfjEzKxAHdTOzAnH6\npcPklYvOY9heK3LY1d5HHu/RY9+t3Tiod5i8ctF5BKJW5LCrvY883qNLCFi7cfrFzKxA3FNvQ/6T\n3swa5aDehoryJ303TK/vhvdonSXXoC7pO8BJwB8i4sA8r2Xtpxv+uuiG92idJe+c+mXAiTX3MjOz\nYZFrTz0ifiVpWp7XKCIPvTOzRjmn3oY89M7MGtXyoD4LZpdeHw9L58KyVrbHzKzdXADTF8KMevZt\neVD/KVzX6jaYmbWzubAs2+EVnFxp35YHdRsZHnpn1h3yHtL4PeBYYGdJy4G/j4jL8rymDc43Rc26\nQ96jX07P8/xmZrY1134xMysQB3UzswJxUDczKxAHdTOzAnFQNzMrEAd1M7MCcVA3MysQB3UzswJx\nUDczKxAHdTOzAnFQNzMrEAd1M7MCcVA3MysQB3UzswJxUDczKxAHdTOzAnFQNzMrEAd1M7MCcVA3\nMysQB3UzswJxUDczKxAHdTOzAnFQNzMrEAd1M7MCcVA3MysQB3UzswJxUDczK5Bcg7qkEyX9VtJD\nkj6Z57XMzCzHoC5pNPD/gBOB/YHTJb02r+u12gUwvdVtaFf+bKrz51OZP5uhy7OnfgTwu4h4LCI2\nAt8H3pbj9VpqIcxodRvalT+b6vz5VObPZujyDOp7AMszyyvSdWZmlpM8g3rkeG4zMxuEIvKJvZKO\nAj4TESemy58CNkfElzL7OPCbmTUgIjTY+jyDeg+wFHgz8CRwJ3B6RDyYywXNzIyevE4cEf2SPgbc\nBIwGLnVANzPLV249dTMzG3meUToMJI2WtEjSda1uS7uR9Jik+9LP585Wt6edSJog6SpJD0pakt6H\nMkDSjPRnpvS1WtLHW92uTpBb+qXLzAGWAL2tbkgbCuC4iHiu1Q1pQ18FboyI09J7UNu1ukHtIiKW\nAocASBoFPAFc29JGdQj31JskaU/gT4FvA4PejTZ/LuUk7QjMjIjvQHIPKiJWt7hZ7eotwMMRsbzm\nnuagPgy+AswFNre6IW0qgAWS7pb0wVY3po3sDTwt6TJJ90r6lqRtW92oNvXnwH+2uhGdwkG9CZJO\nBv4QEYtwb7SSYyLiEGAW8DeSZra6QW2iBzgU+HpEHAq8CMxrbZPaj6SxwGzgR61uS6dwUG/OG4FT\nJD0KfA84XtLlLW5TW4mIp9LvT5PkRI9obYvaxgpgRUTclS5fRRLkbWuzgHvSnx+rg4N6EyLivIiY\nGhF7k/yJuDAi3tvqdrULSdtK6k1fbwecANzf2la1h4joA5ZLKlUhfAvwQAub1K5OJ+kwWZ08+mV4\nedD/1nbSUC5UAAAD3klEQVQFrpUEyc/alRHx89Y2qa38LXBlmmJ4GPjrFrenraQdgbcAvhczBJ58\nZGZWIE6/mJkViIO6mVmBOKibmRWIg7qZWYE4qJuZFYiDuplZgTioW2FIOm6w8seV1g/D9d4m6bWZ\n5VskHVZHG1dLun4Yrj9O0mJJ6yXt1Oz5rBgc1M0adyqwf2a53kkfv4yIk5u9eESsi4iDSR4XaQY4\nqNsIkrSdpBvS3uX9kt6Vrj8s7eXeLelnkqak62+R9K/pQxLul/SGdP0Rkm5Pqxv+OjPVvt42fEfS\nHenxp6Trz5B0jaSfSlomKfuA9PdLWpoe801J/ybpaJJCUxek59kn3f2d6X5LJb2pzjZ9Mn2QyGJJ\nX8y894sk3ZU+ROMNkq5N2/b5et+vdR+XCbCRdCLwREScBCBpB0ljgH8DZkfEs5LeDXwBeD9Jz3d8\nRBySVnf8DnAg8CBJLfJNkt4CfBE4rc42fBr4RUScKWkCcIekBem2g4CDgQ3AUknz0zacT/LAhrXA\nQmBxRPy3pJ8A10XENen7ARgdEUdKmgX8A/An1RqT7ncKcERErEvbRHrd9RHxhvSJPz9O2/A88LCk\niyLi+Trfs3URB3UbSfcBX5b0z8D1EXGbpNcBB5DUXIfkIeXZdML3ACLiV+kvgR2AHYHLJb2GJPiN\nGUIbTgBmSzo3Xd4G2Cs9zy8iYg2ApCXANGAycGtErErX/wjI/mVQXnL5mvT7venxtbwZ+E5ErEvf\n56rMtp+k338D/CYiVqZteCRts4O6vYKDuo2YiHhI0iHAScA/SvoFSTneByLijUM41edJAvCpkl4F\n3DLEprw9Ih7KrpB0JLA+s2oTyf+P8jx5eRAv3146R+n4elSqxV861+aytm0m+eVn9grOqduIkbQb\nsC4irgS+TJJOWApMLj10WdIYSdmbj+9O178JWBURLwA7sKU3P9TKhjcBAw8wTn/JwOCBNYC7gGOV\nPCS6B3gHWwL5mrQtzbgZ+GtJ49P2TKzzOD+UxQbloG4j6UCSHPYi4O+Bf4yIjST58C9JWgwsAo7O\nHLNO0r3A10ny7AD/AvxTun40W/eWBxuBEpn1nwfGpDcmfwN8dpB9thwY8SRJzv5O4DbgUaD0LNHv\nA3Ml3ZO5UVp+3aoi4iaSNMvd6edyTo32131u604uvWttS9L/B86JiHtb3I7tIuLFtKd+DXBpRPy4\nwXMdR/KeZg9j+x4FDouI54brnNa53FM3q+0zaS/6fuCRRgN6aj3wuuGcfESSu/eDzw1wT93MrFDc\nUzczKxAHdTOzAnFQNzMrEAd1M7MCcVA3MysQB3UzswL5X4xGZ/CVpvN+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108574fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEPCAYAAABCyrPIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHY1JREFUeJzt3X20JHV95/H3BwYFBEREWR6Gh8ODioBCFEdBHSMojIrR\nNQpRMKJoRIRg1kP0cDZsTjzBdRNl8OiCCiIqRIgirDxolEFwdVCeYUAeFmTAMKg8CALJIJ/9o6qG\nnjt97+2+09VV3f15nXNP36qurv7OnZn+3k/9flUl20RERKzTdAEREdEOaQgREQGkIURERCkNISIi\ngDSEiIgopSFERARQY0OQtL6kpZKulbRM0j9Os91iSbdJuk7SnnXVExERM5tX145tPyHpdbYfkzQP\nuELSvravqLaRtAjYyfbOkl4BfBFYUFdNERExvVoPGdl+rPz2GcC6wANTNjkIOKPcdimwqaQt6qwp\nIiK6q7UhSFpH0rXACuBS28umbLI1sLxj+R5gmzprioiI7upOCE/ZfinFh/xrJC3sspmmvqzOmiIi\norvaxhA62X5Y0veAlwFLOp66F5jfsbxNuW41ktIkIiLmwPbUX7pn3LiWL2BzYNPy+w2AHwOvn7LN\nIuDC8vsFwM+m2ZfrqnMt/nwnNF1DUccBF4NdfP2dn/7+wIuarq1NP6dRqCs1paYa6nI/29d5yGhL\n4EflGMJS4ALbP5T0IUkfKiu9EPh/km4HTgGOrLGeMbV8MRz3xOrrjrwX7j65mXoiYlTVOe30BmCv\nLutPmbJ8VF01TIYbDZesgEW3wG0vhLsfgbettL9wYdOVRcRoGcoYwpha0nQBEgJOgDd+3H7jOcWg\n/W0/AW6V2Nfmipn3MBRLmi5gGkuaLqCLJU0X0MWSpgvoYknTBXSxpOkCBkHlcaZWk2T3MzAyISQO\nBD4D7GHzVMf69wMH2+zfWHER0bh+PztzLaMR9XQ64H90NoPS14CdJPYdemERMbLSEEbXAcCzgH+d\n+oTNSuAfgL8bdlERMbrSEEbQLOmgkpQQEX1JQxhN06aDSlJCRPQrDWHE9JgOKkkJEdGzNITRM2s6\nqCQlREQ/0hBGSJ/poJKUEBE9SUMYLT2ng0pSQkT0Kg1hRMwxHVSSEiJiVmkIo6PvdFBJSoiIXqQh\njIC1TAeVpISImFEawmiYczqoJCVExGzSEFpuQOmgkpQQEdNKQ2i/tU4HlaSEiJhJGkKLDTgdVJIS\nIqKrNIR2G1g6qCQlRMR00hBaqqZ0UElKiIg1pCG018DTQSUpISK6SUNooZrTQSUpISJWk4bQTrWl\ng0pSQkRMlYbQMkNKB5WkhIhYJQ2hfWpPB5WkhIjolIbQIkNOB5WkhIgA0hDaZmjpoJKUEBGVNISW\naCgdVJISIiINoUWGng4qSQkRATU2BEnzJV0q6SZJN0o6uss2CyU9LOma8uv4uupps4bTQSUpIWLC\nzatx3yuBY21fK2kj4CpJP7B985TtLrN9UI11jILG0kHFZqW0KiXs31QdEdGc2hKC7ftsX1t+/yhw\nM7BVl01VVw2joCXpoJKUEDHBhjKGIGl7YE9g6ZSnDLxK0nWSLpS06zDqaZnG00ElYwkRk632hlAe\nLjoXOKZMCp2uBubbfglwMnBe3fW0ScvSQSUpIWJC1TmGgKT1KH7z/brtNT7sbT/S8f1Fkr4gaTPb\nD3TZ1wkdi0tsL6mh5GFrTTqoZCwhYnRJWggsnPPrbQ+smNV2LAk4A/id7WOn2WYL4H7blrQ38C3b\n23fZzrbHaqyhTAc/A/6XzTlN19NJYj3gVuBQmyuarici5qbfz846E8I+wHuA6yVdU677JLAtgO1T\ngHcAH5b0JPAYcHCN9bRN69JBJSkhYjLVlhAGadwSQpvTQSUpIWL09fvZmTOVm9HadFDJjKOIyZOG\nMGQtnVk0ncw4ipggaQjD1/p0UElKiJgsaQhDNGLpoJKUEDEh0hCGa2TSQSUpIWJypCEMyYimg0pS\nQsQESEMYnpFLB5WkhIjJkIYwBCOeDipJCRFjLg1hOEY2HVSSEiLGXxpCzcYkHVSSEiLGWBpC/UY+\nHVSSEiLGWxpCjcYsHVSSEiLGVBpCvcYmHVSSEiLGVxpCTcY0HVSSEiLGUBpCfcYuHVSSEiLGUxpC\nDcY8HVSSEiLGTBpCPcY2HVSSEiLGTxrCgE1IOqgkJUSMkTSEwRv7dFBJSogYL2kIAzRh6aCSlBAx\nJtIQBmti0kElKSFifKQhDMiEpoNKUkLEGEhDGJyJSweVpISI8ZCGMAATng4qSQkRIy4NYTAmNh1U\nkhIiRl8awlpKOlhNUkLECJu1IUj6a0nPVuErkq6R9MZhFDciJj4dVJISIkZbLwnhcNsPA28ANgMO\nBU6staoRkXTQVVJCxIjqpSGofHwTcKbtG2usZ9QkHUyRlBAxunppCFdJ+j6wCLhE0iYw+2/DkuZL\nulTSTZJulHT0NNstlnSbpOsk7dlf+c1JOphRUkLECJrXwzbvB14K3GH7D5KeC7yvh9etBI61fa2k\njSgayw9s31xtIGkRsJPtnSW9AvgisKDbzqQDL4bli+0bL+zhvWsj7bYI5h8NW28Fz9kBLnkMrm+y\npNaxWSkt/i7ceZ50743wyBNt+LuLiJnN2hBs/1HSCmBXSfMoDiG5h9fdB9xXfv+opJuBrYCbOzY7\nCDij3GappE0lbWF7xZp7vOiNcMSO0m409cFSNINXngRf2unptQ99TtrN+bB7WvFzetWb4dTnAq8t\n1jb7dxcRs5u1IUj6NPAuYBnwx46nftzrm0jaHtgTWDrlqa2B5R3L9wDbAF0aAhQfxH91ksTevb73\nYO3zHjhlxzVrWvRRIB90q8w/Gk7NzylixPRyyOhtwAts/8dc3qA8XHQucIztR7ttMmV5mvRxQvn4\n8+fBqdvDB++aSz1r55nT/Lw22mC4dbTdxut3X5+fU0SdJC0EFs719b00hDuAZwB9NwRJ61HMwPm6\n7fO6bHIvML9jeZtyXRcnlI9X/tT+4F/2W8sgSLctALZb85lHHx96Ma32yBPd1+fnFFEn20uAJdWy\npL5m+/Uyy+hx4FpJp0o6ufxaPNuLJAn4CrDM9uem2ex84LBy+wXAQ93HDyofuAPuPrmHmmuyfDEc\ncfvq65quqY26/ZyOvDc/p4h2kz3z+LCkvyy/rTYUYNtnzPK6fSnGGa7veO0ngW0pdnBKud3nKebz\n/wF4n+2ru+zLcODFcPfJTQ9KFgOm2360OPzx6ONtqKmNVv85bfRceNtK+y17NV1XxCSRZNtTD8tP\nv/1sDaHc6TOBXcrFW2yvnGN9c9LvHyraRWI94FbgUJsrmq4nYlIMvCGUgxRnAL8qV20LvNf2ZXMt\nsl9pCKNP4v3AwTb7N11LxKSooyFcDRxi+5fl8i7A2baHFv/TEEZfUkLE8PX72dnLoPK8qhkA2L6V\n3mYnRaySaxxFtF8vCeF0ihPSvk4xoPxuYB3bh9df3qoakhDGQFJCxHDVcchofeAjwD7lqsuBL8z1\nRLW5SEMYHxlLiBieWmYZNS0NYXwkJUQMz8AagqRzbP+5pBtZ83IStr3HWtTZlzSE8ZKUEDEcg2wI\nW9n+taTt6HK9Idu/6va6OqQhjJekhIjhGNgsI9u/Lr890vZdnV/AkWtZZ0ywzDiKaKdepp2+ocu6\nRYMuJCZO7qoW0TLTNgRJH5Z0A/ACSTd0fN1FbhEWaykpIaJ9ZhpDeDbwHOBE4DieHkd4xPbvhlPe\nqloyhjCGMpYQUa/app1Kej6w6sYntu/uv7y5SUMYX5lxFFGfgV+6QtJBkm4D7gQuA+4CLppzhRGr\ny1hCREv0Mqj8D8ArgVtt7wC8njXvjRwxJxlLiGiPXhrCStu/BdaRtK7tS4GX1VxXTJakhIgW6OWq\npQ9K2pjiGkbfkHQ/8Gi9ZcUksVkprUoJGUuIaEgvF7d7FvAERZp4N7AJ8I1hzjTKoPL4y4yjiMGr\n42qnOwD32X68XN4A2KI8Y3ko0hAmQ2YcRQxWHTfIOZfifgiVp8p1EYOWsYSIBvXSENa1/Z/VQnkf\nhPXqKykmVWYcRTSrl4bwW0lvrRbK739bX0kx4ZISIhrSyxjCTsA3gK3KVfcAh9q+vebaOmvIGMIE\nyVhCxGDUeemKjQBsD33KaRrCZMmMo4jBGOQNcg61faakv2H1O6aJ4gY5/7x2pfYuDWHyJCVErL1B\nzjLasHzceMrXRuVjRJ0ylhAxZDOdqbxj+bjM9reGUUxEJWcvRwzfTAlhkSQBnxhWMRFTJCVEDNFM\nDeEi4EFgd0mPTPn6fS87l3SapBXlnde6Pb9Q0sOSrim/jp/DnyHGVM5LiBiuXqadnm/7oDntXHo1\nxYXwvmZ79y7PLwQ+Ntv+M6g8uTLjKGLuBn7pirk2g/K1l1OkjJnkgz6mlZQQMTzTNgRJPykfH53r\nIaMeGHiVpOskXShp1wHtN8ZLxhIihmDaWUa29ykfN6rx/a8G5tt+TNKBwHnALt02lHRCx+IS20tq\nrCtaJDOOInpTHoZfOOfX9zCGsCNwr+0nJL0O2J1iTOChHgvcHrig2xhCl23vBP7E9gNT1mcMYcJl\nLCGif3Vc/vrbwJPlNY1OAeYD35xjfauRtEU5tRVJe1M0qAdmeVlMoIwlRNSvl1toPmX7SUlvB062\nfbKka3rZuaSzgNcCm0taTvGfeT0A26cA7wA+LOlJ4DHg4Ln8IWJifA04XmLfpISIwevlkNFS4CTg\nk8BbgLuAG2zvVnt1T9eQQ0YB5BpHEf2o45DR4cAC4FO27wS2B86cW3kRay0zjiJq0vPlrwEkbQZs\nY/v6+krq+r5JCLFKUkJEbwaeECRdJmmTshlcBXxZ0mfXpsiItZSUEFGDXg4ZPdv274G3U0w33RvY\nr96yIqaXGUcR9eilIawraUvgncD3ynW9H2eKqEdSQsSA9dIQ/h64BLjD9pXliWq31VtWxMySEiIG\nr69B5aZkUDm6ydnLETMb2D2VO3a4AfB+YFdgg3K1bR8+5yr7lIYQ08mMo4jp1XEewpnAFsABwBJg\nG4p7HES0QcYSIgakl4Rwre2XSrre9h6S1gOusP2K4ZSYhBAzS0qI6K6OhPCf5ePDknYHNgWeN5fi\nImqSlBAxAL00hC+VJ6UdD5wPLAP+Z61VRfQhM44iBiOzjGIsZMZRxJoGNstI0t90WW2KeyDb9j/P\nrcT+pSFELzKWELG6QY4hbAxsNOVr447HiLbJWELEWsghoxgrSQkRT6vjaqdnSNq0Y/k5kk6ba4ER\nNUtKiJijXmYZvcT2Q9WC7QeBveorKWLuMuMoYu56aQgqp51WC5sB69ZXUsRaS0qImIN5PWzzT8BP\nJX2LYobRnwOfqrWqiLVgs1JalRIylhDRo54GlSW9GPhTimmnP7K9rO7Cprx/BpWjLzkvIaKeQeUd\nKe6FcDJwE7Bf5yBzRBtlLCGif72MIXwbeFLSTsApwHzgm7VWFTEYGUuI6EMvDeEp209S3FP5ZNsf\nB7ast6yItZeUENGfnq52KukvgMOA/yNJwHr1lhUxMEkJET3qpSEcDiwAPmX7TmB7ipvmRLReUkJE\n7/q6dEV5DsI2tq+vr6Su75tZRjFnmXEUk6qOWUaXSdqkbAZXAV+W9Nm1KTJimJISInrTyyGjZ9v+\nPcWg8tds7w3s18vOJZ0maYWkG2bYZrGk2yRdJ2nP3sqO6FvGEiJm0UtDWFfSlsA7ge+V63o9znQ6\ncMB0T0paBOxke2fgg8AXe9xvRF+eTgkXLJYOvFh655LicbdFTdcW0Ra9XLri74FLgJ/YvrI8Ue22\nXnZu+3JJ28+wyUHAGeW2SyVtKmkL2yt62X9Ef15+P7x+d7io49/9ETtKu2HfeGFzdUW0w6wJwfY5\ntvew/eFy+Q7b/3VA7781sLxj+R5gmwHtO2KKzT8CJ075JehLO8G2H22mnoh2mTYhSDrO9qclnczT\nt86s2PbRA6ph6gh418NRkk7oWFxie8mA3j8mxsbrd1+/0QbDrSOiHpIWAgvn+vqZDhlVF7C7qstz\ng7rN2r0Ul8KobFOuW/MN7RMG9J4xsR55ovv6Rx8fbh0R9Sh/UV5SLUvqa2bdtA3B9gXl41fnVlpP\nzgeOAs6WtAB4KOMHUZ/li+GIHYvDRJUP3AF3n9xcTRHtMe2JaZIuYM1DRRXbPmjWnUtnAa8FNgdW\nUMwDX6/cwSnlNp+nmIn0B+B9tq/usp+cmBYDUcwq2vajsN0OsMHm8P3DMqAc46rfz86ZGsJvKAZ5\nzwKWVqvLR9u+bG0K7UcaQgxaefbyL4HDcvZyjKtBNoR5FHebOgTYneIchLNs3zSIQvuRhhB1kDgc\nOMTOXdViPA3s0hW2n7R9ke3DKC5udztwmaSjBlBnRBucCeyYs5cjCjNe3E7S+sCbgIMprnJ6PnCa\n7a4zgeqShBB1SUqIcTbIQ0ZnAi8GLgT+xfa01yOqWxpC1CVjCTHOBtkQnqKY+dONbW8yh/rmJA0h\n6pSUEONqYA2hTdIQok5JCTGuBn4/hIhxl/slRBTSECIKmXEUEy8NIYKkhAhIQ4jolJQQEy0NIaKU\nlBCTLg0hYnVJCTGx0hAiOiQlxCRLQ4hYU1JCTKQ0hIgpkhJiUqUhRHSXlBATJw0hooukhJhEaQgR\n00tKiImShhAxjaSEmDRpCBEzS0qIiZGGEDGDpISYJGkIEbNLSoiJkIYQMYukhJgUaQgRvUlKiLGX\nhhDRg6SEmARpCBG9S0qIsZaGENGjpIQYd2kIEf1JSoixVWtDkHSApFsk3SbpuC7PL5T0sKRryq/j\n66wnYm0lJcQ4k+16diytC/wS2A+4F/g5cIjtmzu2WQh8zPZBs+zLtlVLoRF9kliP4t/2YTZXNF1P\nxHT6/eysMyHsDdxu+y7bK4Gzgbd22S4f9DFSkhJiXNXZELYGlncs31Ou62TgVZKuk3ShpF1rrCdi\nkDKWEGOnzobQy7Goq4H5tl8CnAycV2M9EQOTlBDjaF6N+74XmN+xPJ8iJaxi+5GO7y+S9AVJm9l+\nYOrOJJ3QsbjE9pLBlhvRtzOB4yX2zVhCtEE5Lrtwzq+vcVB5HsXA2+uBXwNXsuag8hbA/bYtaW/g\nW7a377KvDCpHK0kcDhxis3/TtURM1ZpBZdtPAkcBlwDLgH+xfbOkD0n6ULnZO4AbJF0LfA44uK56\nImqSsYQYG7UlhEFKQog2S0qItmpNQoiYIEkJMRbSECLWUmYcxbhIQ4gYjKSEGHlpCBEDkJQQ4yAN\nIWJwkhJipKUhRAxIUkKMujSEiMFKSoiRlYYQMUBJCTHK0hAiBi8pIUZSGkLEgCUlxKhKQ4ioR1JC\njJw0hIgaJCXEKEpDiKhPUkKMlDSEiJokJcSoSUOIqFdSQoyMNISIGiUlxChJQ4ioX1JCjIQ0hIia\nJSXEqEhDiBiOpIRovTSEiCFISohRkIYQMTxJCdFqaQgRQ5KUEG2XhhAxXEkJ0VppCBFDlJQQbZaG\nEDF8SQnRSmkIEUOWlBBtlYYQ0YykhGidNISIBiQlRBvV2hAkHSDpFkm3STpumm0Wl89fJ2nPOuuJ\naJmkhGiV2hqCpHWBzwMHALsCh0h60ZRtFgE72d4Z+CDwxbrqGTRJC5uuYarU1Ls21DU1JbShpqlS\nU2/aWNNc1JkQ9gZut32X7ZXA2cBbp2xzEHAGgO2lwKaStqixpkFa2HQBXSxsuoAuFjZdwDQWNl1A\n6Uz4wW7SwUthp69KB14s7bao6aKk3RZJB16cmkavJuisqz91NoStgeUdy/eU62bbZpsaa4pomd32\nhwvWgbP3hndvBxe9EV55UpMfLMV7v/KkopbUNEo1rVlXf+bVUVDJPW6nOb4uYgzMPxoWP3/1dV/a\nCY79qsTSZmra7xXwueelplGsCbrX1RvZ9Xz+SloAnGD7gHL5E8BTtj/dsc3/BpbYPrtcvgV4re0V\nU/aVJhERMQe2p/7SPa06E8IvgJ0lbQ/8GngXcMiUbc4HjgLOLhvIQ1ObAfT3B4qIiLmprSHYflLS\nUcAlwLrAV2zfLOlD5fOn2L5Q0iJJtwN/AN5XVz0RETGz2g4ZRUTEaGn1mcqSTpO0QtINTddSkTRf\n0qWSbpJ0o6SjW1DT+pKWSrpW0jJJ/9h0TRVJ60q6RtIFTdcCIOkuSdeXNV3ZdD0AkjaVdK6km8u/\nvwUtqOkF5c+o+nq4Jf/WP1H+37tB0jclPbMFNR1T1nOjpGMaqmGNz0pJm0n6gaRbJX1f0qaz7afV\nDQE4neLEtjZZCRxr+8XAAuAjU0+4GzbbTwCvs/1SYA/gdZLacvbrMcAy2jN7zMBC23va3rvpYkon\nARfafhHF39/NDdeD7V+WP6M9gT8BHgO+02RN5XjkEcBetnenOBR9cMM17QZ8AHg58BLgzZJ2bKCU\nbp+Vfwv8wPYuwA/L5Rm1uiHYvhx4sOk6Otm+z/a15fePUvzn3arZqsD2Y+W3z6D4j/JAg+UAIGkb\nYBHwZdacXtyk1tQi6dnAq22fBsXYm+2HGy5rqv2AO2wvn3XLev2e4heyDSXNAzYE7m22JF4ILLX9\nhO0/ApcBbx92EdN8Vq468bd8/LPZ9tPqhtB25W8se0KTc44LktaRdC2wArjU9rKmawI+C3wceKrp\nQjoY+DdJv5B0RNPFADsAv5F0uqSrJX1J0oZNFzXFwcA3my7C9gPAPwF3U8xcfMj2vzVbFTcCry4P\nz2wIvIn2nFy7RceszRXArFeBSEOYI0kbAecCx5RJoVG2nyoPGW0DvKbpa6tIejNwv+1raNFv5MA+\n5WGQAykO97264XrmAXsBX7C9F8Vsu1mj/bBIegbwFuCcFtSyI/DXwPYUqXwjSe9usibbtwCfBr4P\nXARcQ7t+AQLAxeyhWQ/bpiHMgaT1gH8Fvm77vKbr6VQebvge8LKGS3kVcJCkO4GzgD+V9LWGa8L2\nv5ePv6E4Jt70OMI9wD22f14un0vRINriQOCq8ufVtJcB/9f272w/CXyb4t9Zo2yfZvtltl8LPAT8\nsumaSisk/RcASVsC98/2gjSEPkkS8BVgme3PNV0PgKTNqxkEkjYA9qf4TaUxtj9pe77tHSgOOfzI\n9mFN1iRpQ0kbl98/C3gD0OgMNtv3Acsl7VKu2g+4qcGSpjqEoqG3wS3AAkkblP8P96OYsNAoSc8v\nH7cF3kYLDq+VzgfeW37/XmDWX17rPFN5rUk6C3gt8FxJy4H/bvv0hsvaB3gPcL2k6kP3E7b7vrLg\nAG0JnCFpHYomf6btHzZYTzdtmGW0BfCd4rOEecA3bH+/2ZIA+CjwjfLwzB205ATNsmnuRzGzp3G2\nrytT5i8oDstcDZzabFUAnCvpuRQD3kfa/v2wC+j4rNy8+qwETgS+Jen9wF3AO2fdT05Mi4gIyCGj\niIgopSFERASQhhAREaU0hIiIANIQIiKilIYQERFAGkJMAEmPlo/bSZp617613fcnpyz/ZJD7jxim\nNISYBNXJNjsAf9HPC8uras7kE6u9kb1PP/uPaJM0hJgkJ1JcmfKa8qYm60j6jKQrJV0n6YMAkhZK\nulzSdymuZomk88orpN5YXSVV0onABuX+zizXVWlE5b5vKG/I886OfS+RdE55Q5yvV8VJOrG8+ct1\nkj4z1J9MBC2/dEXEgB0H/DfbbwEoG8BDtvcu77x1haTqUhZ7Ai+2/aty+X22HyyvFXWlpHNt/62k\nj5RXT61UaeTtFDdM2QN4HvBzST8un3spsCvw78BPJO1DcZ2eP7P9wrK2TWr480fMKAkhJsnUy3C/\nATisvCbVz4DNgJ3K567saAYAx5T3m/gpMB/YeZb32hf4pgv3U9w45eUUDeNK278uL0l8LbAdxVUy\nn5D0FUlvAx6f858yYo7SEGLSHVXdKtL2jh03XPlDtUF5b4nXAwvKe05cA6w/y37Nmg2oSg//0bHu\nj8B65d229qa4/PWbgSYvlhgTKg0hJskjwMYdy5cAR1YDx5J2meZuZZsAD9p+QtILKe6lXVk5zcDz\n5cC7ynGK5wGvAa5kmpsFlVcW3dT2RcDHKA43RQxVxhBiElS/mV8H/LE89HM6sJji7ltXl9fXv5/i\nevZT7y51MfBXkpZR3Pzkpx3PnUpxKfSrbB9avc72dyS9snxPAx+3fb+kF7HmpcBN0ai+K2l9iqZx\n7ED+5BF9yOWvIyICyCGjiIgopSFERASQhhAREaU0hIiIANIQIiKilIYQERFAGkJERJTSECIiAoD/\nD7SICfBEbeEDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x107ab5320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "\n",
    "ppn = Perceptron(epochs=10, eta=0.1)\n",
    "\n",
    "ppn.train(X, y)\n",
    "print('Weights: %s' % ppn.w_)\n",
    "plot_decision_regions(X, y, clf=ppn)\n",
    "plt.title('Perceptron')\n",
    "plt.xlabel('sepal length [cm]')\n",
    "plt.ylabel('petal length [cm]')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(1, len(ppn.errors_)+1), ppn.errors_, marker='o')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Misclassifications')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the perceptron converges after the 6th iteration and separates the two flower classes perfectly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problems with Perceptrons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although the perceptron classified the two Iris flower classes perfectly, convergence is one of the biggest problems of the perceptron. Frank Rosenblatt proofed mathematically that the perceptron learning rule converges if the two classes can be separated by linear hyperplane, but problems arise if the classes cannot be separated perfectly by a linear classifier. To demonstrate this issue, we will use two different classes and features from the Iris dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD7CAYAAAClvBX1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt0FeW9//H3NyEkXFKCgBFJ0KqAN5CoP8oRaa29KKho\na9pf/Z1WKVatHIxHKqu0emrbJcfTSq3RWosWj1T96aL4K8qp1PYUe7wtseANhIKgyP1ONBASc3l+\nf+zZutnsZM8Os/eeST6vtVjds+fJzHee4jfDM9/nGXPOISIi0VKQ7wBERCRzSt4iIhGk5C0iEkFK\n3iIiEaTkLSISQUreIiIR1CNXJzIz1SSKiHSCc86Sv8tZ8gZwcF0uz3ckJsAli2FRvuPwK2rxQvRi\nVrzZF7WYcxGvwZxU32vYREQkgpS8RUQiSMm7HefDmnzHkImoxQvRi1nxZl/UYs5nvJartU3MzEVp\nzFtEJAwM5uT9gaWIyJFqgIJ7i4tP/sfQoad/VFjYC7PDEltkOOd6trYePHnjxpU3NDX9oze0+f1R\nJW8RiZQfDBjw2eMmTDj+jgkTPhz0qU/tLSwszHdIndba2squDz8s+r+LF4/5weLFR9fu2fM3vz+r\nMW8RiZQ9gwYNnVpdvfeY/v2bo5y4AQoLCzmmf//mqdXVe/cMGjQ0k59V8haRSHFmBSU9e3apSX8l\nPXs6Z5ZRPlbyFhGJICVvEZEI0gNLEZEseWnVqtKbHnjg1DWbN/erb2jo2fbMM/8V1LF15y0ikiUl\nRUVtl48bt/Wua655M+hj685bRLqNMddcM77hwIGe8e3effp89OqDD76QrfOdNWzYgbOGDTvw+vr1\nvYM+tpK3iHQJyYkZDk/ODQcO9FzZp89H8e3Tk9pHiZK3iHQJyYkZMk/Ofn4BhIWSt4iI50h/Afzy\nD38Ycsu8eaMARp9wwp6X77rr1aBjjFPyFpFuo3efPh+dnjTmHeTxb/rKV7bc9JWvbAnymO1R8haR\nLiE5Mce/S9zOx/DH/oMHC5qamwsAGhobCwB6l5T4XoCqPUreItIlBJGY/fwCyMTK99/vNer6678A\nYGb0/epXJ5b379+w7bHHlhxprEreIiKeoO/MTz/uuINBTsxJpEk6IiIRpOQtIhJBSt4iIhGk5C0i\nEkEdJm8zKzGzpWb2hpmtMrM72ml3j5m9Y2ZvmllVdkIVEZG4DqtNnHONZvZ551yDmfUAXjSzc51z\nL8bbmNlE4CTn3DAz+wxwPzA2u2GLiHRvaYdNnHMN3seeQCGwN6nJJGCe13YpUGZm5UEGKSIih0qb\nvM2swMzeAHYAzznnViU1GQJsStjeDFQEF6KIiCRLO0nHOdcGjDazfsCzZnaec+5vSc0s+cdSHWsC\nXBL/fD6smQFrM4xXRKRLuxOGL4ER6dr5nmHpnPvAzP4InA38LWHXFqAyYbvC++4wi2GR3/OJiETN\nvz3yyPHzn3++8r0dO0ovOPPMLYt+/OOM36AzA9Ym3tgaXJyqXbpqk4FmVuZ97gV8CXg9qdnTwJVe\nm7FAnXNuR6YBi4hk0wcNPQo37Oj18bol67b2Lm5oLAi0XLpy4MDGGdXVay8dO3ZT+tZHJl3gg4El\n3pj3UmCRc+6vZnadmV0H4Jx7BnjXzNYBc4CpWY1YRCSF97b3Km5rO3Q7cf8zfx/U/1uzR52xfluv\n4tUb+/S6cvaoM/77jYFlQcZw7YQJ279zwQU7jurbN9ClZlNJVyq4AjgzxfdzkranBRyXiIhvbW3w\nvd+ePOLUofs//Ok339kw988Vxzzy1yEVz97+9+W9itscwBWf27Z7Z13PnhN/dPYYgJsvf++dSWN3\nflw9V99QWPDTx086/pZvrH+/rE9L68r3+/b6w8vlg/7tivUbM40n5UO/gGmGpYhEXkEBzLlh5aq/\nr+3Xf8y/njP2P/9cMXTODSvfjifuuC9X7d4X/3z+GXv2Je7rXdza1tRcUHDVL0ad9vLqsr7fuXvk\nqKGDGhs7E09yBUc2KHmLSJcwqF9zyxdH79lVf7CwaETFgfoRFQcOJu5fvbFPr6vvHjny5svfe2fq\nxRvXT75r1Kj12z4ZWikshF9es3qdAd++a2TVlV/YsvGqL27Z2ZlYdOctIuLTg3+qOOYPL5cf+9BN\nK97Yure45EePDjs+cQx8+77ioqsv2Pz+NRdu3n7jpe9vveJz2zbvqCsuSjzG6s19e23bV9yrqNC1\nPfvawEF1B3oUdiYW3XmLiPjQ1gbvbu/da+6/rlgx7tS6+kdnvLli/8EePZqaCz7Oo58/Y++HV1+w\n+eNKuKkXb9x2zil1++PbHzT0KLy29vSRUy/a+N6b97340nFHH2y44f5T09ZbJ2puaWH/wYMFLW1t\n1trWZg2NjQXNLS3BXGQSvUlHRCKvoADumLz2vfj2oH7NLXdft3pdJsfo17uldd733npr2JCGRogN\noby7vXdJJseo+c1vhs955pnh8e2+X/1qxfUXX7zmvqlT38nkOH4oeYuIeOKJG2Jj4Inbftw/bdra\n+6dNy8nMcQ2biIhEkJK3iEgEKXmLiESQkreISAQpeYtIpJhzbY0ffZSLUuqcafzoI7PY8tu+KXmL\nSKQM2LVr430LFhy1be/eotbW1nyHc0RaW1vZtndv0X0LFhw1YNeujNZQMedyMZETzMw5uC4nJxOR\nLqsBCu4rLh6xurJyZFOPHr0wi+5duHOuuKXl4CmbNq34l6amNb3hsLtvgznOucOuUclbRCTE2kve\nGjYREYkgJW8RkQhS8hYRiSAlbxGRCFLyFhGJICVvEZEIUvIWEYkgJW8RkQhS8hYRiaAOk7eZVZrZ\nc2b2tpmtNLOaFG3OM7MPzOx178+t2QtXREQg/WvQmoGbnHNvmFlfYLmZ/cU5tzqp3f845yZlJ0QR\nEUnWYfJ2zm0Htnuf95vZauBYIDl5R3dhGJEQGQT39kj477IFWnbBDUGfZwT8sBVK49uFUL8G/j3o\n80j2+H4BsZkdD1QBS5N2OeAcM3sT2ALc7JxbFVSAIt1JD+ixDVri24Oz9JLwVihdB/Xx7ZMSErlE\ng6+/GN6QyQLgRufc/qTdrwGVzrkGM5sALASGpzrOBLgk/vl8WDMDcvKWZRGRqLgThi+BEenapU3e\nZlYEPAk86pxbmLzfOVef8Hmxmf3azI5yzu1NbrsYFqUPXUSk+5oBaxNvbA0uTtWuw+RtsUXO5wKr\nnHN3t9OmHNjpnHNmNobYGuGHJW4RSa8FWgYnjXln4zyFUH9S0ph3Ns4j2dPhyxjM7FzgeeAtYmPb\nAD8EhgI45+aY2b8A1xP7S9YATHfOvZLiWHoZg4hIhvQmHRGRCGoveWflSbZIGOSyHC5X51KJn8Qp\neUuXlctyuFydSyV+Eqe1TUREIkjJW0QkgjRsIl1WLsvhcnUulfhJnKpNRERCTNUmEilhqqrws1iU\nn3j7w2+KExZxawK3D76bSSxh6hfJLyVvCaUwVVX4WSzKT7zFYNs/mezGMZ1YjTNM/SL5pQeWIiIR\npOQtIhJBGjaRUApTVYWfxaL8xNsE7pikMe9MYwlTv0h+qdpERCTE2qs20bCJiEgEadhEuqygyupU\nntc+9U3+KHlLlxVUWZ3K89qnvskfDZuIiESQkreISARp2ES6rKDK6lSe1z71Tf6oVFBEJMS0MJV0\nO0FVQvhZUEqVLZJrSt7SZQVVCeFnQSlVtkiu6YGliEgEKXmLiERQh8MmZlYJ/A44mtg/Gx9wzt2T\not09wASgAZjsnHs9C7GKZCSoSgg/C0qpskVyrcNqEzM7BjjGOfeGmfUFlgOXOedWJ7SZCExzzk00\ns88Atc65sSmOpWoTEZEMdaraxDm3Hdjufd5vZquBY4HVCc0mAfO8NkvNrMzMyp1zOwKLXkSk+5hS\nxbLRn2yenbKR72oTMzseqAKWJu0aAmxK2N4MVABK3pJSrsrh/Lx70g+VAbavK15TnkyppeZEgNlM\nL/uAsrp6+v3J2/fNVD/gK3l7QyYLgBudc/tTNUnazs3MH4mkXJXD+Xn3pB8qA2xfV7ymHJiSuFFL\nzYkJCbueWH/+R0KTziVvMysCngQedc4tTNFkC1CZsF3hfXeYCXBJ/PP5sGYGrE13fhGRLmA8QC01\nF85melkTJY3xHY2UNNbTbznwUOybO4fDkktSHiVBumoTA+YCq5xzd7fT7GlgGvCEmY0F6tob714M\ni9IFJCLSldRSM2s208vaKGwF2EpFQqJOZcba2J84uzhVq3R33uOI3bK/ZWbx8r8fAkMBnHNznHPP\nmNlEM1sHHAC+7eeCpPvKVTmcn3dP+qEywPZ1xWsKwPhq5l8GsJ4TinZS7iNhZ04LU4mIHJmZiRuV\nbKhsoqRxF+Vve18dYdI2LUwl4RBEhUIuF4sSSTIeGAZQxbLROymnntKP/8VRT79NHPrAMSuUvCXn\ngqhQyOViUSIJZsbvrAEaKamvp98t+QhEyVtEJLWkyTKwm4GFmzkuJ3fW6Sh5i4h8YmYtNaWQcrJM\n3At5iOswSt6Sc0FUKORysSjpsqYQW3QPgGrmD1rKmBLvzhpgPQFXiARJ1SYiObCcM0tXMLJsMvM2\nATzMVZUjWVF3Fq/pF0puHTJZ5gPK6hJ3enfZobiz/oSqTSQkgqgC8XMMP238rH/SH+YUJ2w3AfuS\nbkTSnWsFI8umMuXmW1ndBLCHyb0H8f3dG+HHmVy3dNr4auZftpQxJf4ny4SbkrfkXBBVIH6O4aeN\nn/VPivGW1vQc04l4JjNv062sbhrAA70Afs93d3+LV4uTjyOB+niyTDxpb6XiCUJ3Z905St4i0hWM\nr6XmwsQv4muIBDdZJlyUvEVy4GGuqtzD5N6/57u7Ab7Pz/o3Mr0p9n4T6aT46nxHJ8xq3JWwP3l1\nvi5FyVtyLogqED/H8NPGz/onTRw6VNLUiXhGsqJuEN//eKikkelNxWz9sN0LlHQOmSyzi/JQ1F7n\nkqpNRCTsZlax7NjEL8I0WSb7VG0ikjGV+OVHLTWzAF7g3JKE2uuXkpp1iQePnaXkLTkXpQWjVjCy\n7CfcVt1K4ZMAt3Pr5bfxkwVK3oE6ZFW+hMky8QcC79DNE3UqSt6Sc1FaMGoy8za1UvjkLcz6JsAs\nbnk0fhcuR2Q8QLz2OmmyTGM9/RaihN0hJW8RyaXxVSz72m4GFiZMlukytde5pOQt0oGHuarydm69\nfBa3PAqxYZNCWhfo7jsjU6qZfxrEJst4JX01+Q4q6pS8JeeitGDUSFbU3cZPPk7WhbQuGMmKunQ/\n141NqaXmxMQvElbnW+991aUmy+SLSgUlcrpqBUi66wrxdaebLLMTJewjoFJB6SJyXQGSq+qYdNcV\nxsqX+JvRu/NkmXxR8pbIyXUFSK6qY9JdV74rX2qpmfUwVx5y7d5kmUivzhdVSt4i0q5aama9wLkl\nEHvY6CXqdxKaqEokT5S8JXK6agVIuuvKwXUfMlmmimXH7qScrVS84X2lyTIhkjZ5m9lDwEXATufc\nyBT7zwOeAt71vnrSOXd7kEGKJMp1BUiuqmPSXVeWrns8QLz2OmmyTN7ejC7ppa02MbPxwH7gdx0k\n7+nOuUlpjqNqE5HwmFLFstE7KQegmaLmXZR/L88xSUqpq00K0v2Yc+4FYF+6o3c2LMm/5ZxZ+jBX\nVca3H+aqyuWcmZWHcrk6l5/z+Gkzm+knTWLhBfHtSSy8YDbTT0ps8wRfH3wDtWfHt2+g9uwn+Prg\nTM+VAzOrWHZPFcvuqWTDWes5qX4rFTVbqahR4o6eIMa8HXCOmb0JbAFuds6tCuC4kiO5fL9irsrd\n/FyTn1ie57Mn/hcXX3YRiwoAFjNhErDwZu5aF2/zEuOGPMi1/9xKjwKAh5hyBfDYN5i/LdfXnaCj\nyTJ/8r7S+HWE+ZqkY2bHA4vaGTYpBVqdcw1mNgGodc4NT9FOwyYhVsGYn8ffr/gbvrvnW7xSvA5+\nkI1zzWXK0MRyt6t5aGM2zuPnmvzEchGLJjzDRZcCTOSPT/2RSxYnt5nKfWMe5JpvAVzDg4/8mn95\nNblNDq57CkAtNScmJOrEXw6aLBNJWZqk45yrT/i82Mx+bWZHOef2JredAJfEP58Pa2bA2iM9v4hA\nNfN/EV83BGAX5aq9jqw7h8OSEelaHXHyNrNyYpUozszGELubPyxxAyyGRUd6PgleLt+vmKsyPz/X\n5CeWSSy8YDETJk3kj09BbNhkEgvbnuayZ+NtbqD27IeYcsU1PPgIxIZNCmlpu5cbl2XjulNNlvFK\n+rQ6X5cwY23sT5xdnKqVn1LBx4HPAQPNbBNwG1AE4JybA1QD15tZC9AAfONIQ5fcyuX7FXNV5ufn\nmvzE8lmeXw8sjCfrSSxs87772Dhe2gI8Fk/WhbS0ed9ldK6OVDP/FwDrOaEoofZad9bdWNrk7Zy7\nIs3++4D7AotIcu4sXqs/9OFk9t5ofhav1Sc+pOvMnaffBZp+yilzJ/Oq1+b0ypG4OtiW6pDtqmDz\ngeN4f098+zje31PB5gOJbYaxbn8jvXbEt8/itR3DWLc/0+tKcMhkmUo2VLZR2JowWUZJWzTDUsKp\no8Wg/FRuBNUmqEqSNG3Ge82GxWuv6ylN/EVUp8kykkzJW0Kpo8Wg/CzQFFSbe7lxWSs9ChIrSRLH\nsgM415RKNpwVf6tMPaWa1Si+KHmL5Fg5Owb/nJtn/IqpLj4lXQlbMqXkLZHjp3IjqDYBVJLMrKWm\ndCOVPZ7i0j6nsrL5bU4r2MngZmAJsdnLqhCRjCl5Syh1tBiUn8qNoNp0ppJkBGuaj+P9idXMH+4t\no7opdgkcWM+wj4CXgA1AHSF+BZyEm5K3+PIEXx/8EuOGxJPYDdSePY6XtiQ+uPPDb6XIkb6p5jnO\nK3+ez54YP8//4yun7mbA+sTz3M+1p77COadPZt5cgNl878tjeXnlb/nu0ngbP5UkCRU046uZf5km\ny0guKHmLL36qLvwIYo2PoNYkeYVzTn+b08ecwtsFAP/glPjiUkv9nCs+qzExNu/9jZosI1mnFxCL\nb37W7/AjiDU+glqT5BTeviaetE9m9bLVnPZge+fqy/7SoWwoOYp9bUsZU+LVXv8eJWrJKr2AWCQT\n44FxAKeyqmc5Owat5uSiPfQ/WMeAt702Gg6RvFHyFl/8VF34EcQaH0GtSXI6b139D045+2RWL4PY\nsMl5LBnwN87fWcmGs5ooaTxAn5YmSvoU03igmeIDdRTXA38BIv3KNYk+JW/xxU/VhR9BrG0S1Jok\nY3l5JcBKRs0FZvaivuV1Rh07hE3H1lMar70uBcoa6BtP1pXEqkRE8kpj3tJtJa7Ot5uBhV5J33/k\nNyqRZBrzlohIV5bop9wwuc1Sxvzoz3y55FN86GI/c2Wpl7CXE3srOqR+8FgKlPHJMEn8zlv12ZJX\nSt4SOunKEv0uBDWH62pO4N2NP+ZHJ7zMuF5N9NoKfOQ1qSc2WSZdpUgZsWWPn/S2LwcWoOQteaZh\nEwmldGWJ7ZQKxlfno4plX9tMhdVxVA+AFor+BDzVyXCGAt/0Pj8KZOW1bSKpadhEuqC+7C/9OTfP\n+DE/KoivzAfQTFHzbspr+STpvp6fCEWyQ8lbQiddWeKLjJv2A+449XTeanqb03ruY4A10/NPwNMJ\nh6kkNtzxqLcdH+7ItMSv0vvZIz2OSKCUvCV0ksoSx/8v/r5/FG9OAL4I8HUerzxA7/rn+OIa70f+\nwuHle3UcmmQXpGjjR1DHEQmUxrwlMH6qQHwucBUfux5XyYbKJkoaG72FngDq6VePSvqk29CYt2RZ\nEK8Vq6Vm1myml7VR2NpMUfMuylV7LZKC7rwlUH4WjIpXkhTRXDSIHS1Hsbctvi+h9lrrhogAuvOW\nUKilZlYd/Yod1rOREjby6X0b+fSfk5pplT6RNJS8JTAdLBj1JYAqlo3ewPEFdQwoBHYCa4EzgYNA\nRgtciXR3aYdNzOwh4CJgp3NuZDtt7gEmAA3AZOfcYTW1Gjbp+pZzZukeBnz+y/xlH8CLjDvj6zx+\nimGtzRQ1N1LSWE+/XwFD+CRZnw1sATJ6qYNI99H5YZP/BO4FfpfysGYTgZOcc8PM7DPA/cDYIwlV\nImnKd3hg9E7KD/ly/yer8yVKTNS64xbphLTJ2zn3gpkd30GTScA8r+1SMyszs3Ln3I4OfkYC4Pd9\nkEEcJ1WbL/DXiTdx13CApYwp2cNRHx6k9EkOv6vOVLrFoLRYlHR7QYx5D+HQ2WabgQpAyTvLgngf\npN/jHKTXVXO5+qQRrNkba3PL0d/nDjtI77p6+sXXyX4L+GegwNu+AniMzIdE0i0GpcWipNsL6oFl\n8nhMbuoPu7nJzNvUSuGTiaV5mbyRxsdxpgDUUnPibKaX7WXA/nN5sRyghIP7G+i7hsNL+gqAb3mf\nH6FzwyKbiCXmxMWgNmWwX6TLCyJ5byH2z9a4Ctr5p/IEuCT++XxYMyNWbSAhFJ8s0+TNbNxF+XLg\nv/ESZgN9tbqeSFbcORyWjEjXytckHW/Me1GqahPvgeU059xEMxsL3O2cO+yBpapNgvcwV1X+hNuq\nb+X2Q4Y7Mr37/h53/uwBrv3UsWxpAdjKkB4lHGzZTfnrfHJnHV/oKXmoIvFcZxMbNnnc244Pm2R6\n953uXH5iEekiUleb+CkVfBz4HDCQ2Dj2bUARgHNujtfmV8CFwAHg286511IcR8k7YEfywLKWmlmx\nn7mydBuD2cGxa4E3vd39gec4dAzZz0PCwQRTBqgHliIf62TyDuz0St75Nh4YB1DFsmN3Us5WKrZ6\n+7R2iEhoaXp8dzI+aTt5db76FLXXIhIhSt5dz5RKNpyV+FYZgK1UaHU+kS5EybtrmFnN/EEQmyzz\nAWWpZjWKSBei5B09MxM3qpk/aCljSjZz3CZiiz2BllMV6fKUvKPhkMkyXqKO2wW8hJZRFelWlLzD\nbXw18y9bypiSpMkyurMW6eaUvEOkmvm/WM8JRfHt3QwsbKOwdSsVT6A7axFJoOSdX+NrqbkQYpNl\nvNrr3ye1UdIWkcMoeefeFOBogHjt9S7KdwHr0XCIiPik5J1dKSfLfEBZHUA9/VR7LSKdouSdPTMr\n2VCpyTIikg1K3gGqpWbWC5xbArHJMl5JnxK1iAROybvzZtZSUxrfeIFzS7yEvdz76h30sFFEskTJ\nOzNTAKqZf1rCnXVcPZosIyI5ouTtz/gqln1tNwMLEybLqPZaRPJGyftw46uZf1nyZBmvpK8mn4GJ\niMQpecdMqWb+aRB70Oglak2WEZHQ6s7JO9Vkmbe9fZosIyKh1l2S9yGTZWqpuXA208s0WUZEoqrL\nJ+9aambNZnpZiskyWp1PRCKrSybv+GSZ9ZxQtJuBhV7ttRK1iHQZkU/etdTMStxOWJ3vDWITZUAP\nG0Wki4lq8p4CUMWy0d6ddfJkGY1fi0iXljZ5m9mFwN1AIfBb59zPkvafBzwFvOt99aRz7vaA44yb\nUsWy0Tspp5miZqBZtdci0h11mLzNrBD4FfBFYAvwdzN72jm3Oqnp/zjnJgUc25Qqlo1O/GI3Awv1\nZnQRkfR33mOAdc65DQBm9gRwKZCcvC2geFJNllmYsF9j1yIipE/eQ4DE8eTNwGeS2jjgHDN7k9jd\n+c3OuVUZxDAToJaa0njtdT391nv7VCEiIpJCuuTtfBzjNaDSOddgZhOAhcDwdtoeMlkm/mb0hMky\nKukTEfEhXfLeAlQmbFcSu/v+mHOuPuHzYjP7tZkd5Zzbm3ywT1HzfxzmAIr5p7Zi/qlNb0YXEUl0\n53BYMiJdK3Ou/ZtrM+sBrAG+AGwFXgWuSHxgaWblwE7nnDOzMcB859zxKY7lwC1Fd9YiIhmwOc65\nw54rdnjn7ZxrMbNpwLPESgXnOudWm9l13v45QDVwvZm1AA3ANzo4pBK3iEgAOrzzDvREsTvv63Jy\nMhGRLiP1nXdBPkIREZEjo+QtIhJBSt4iIhGk5C0iEkFK3iIiEaTkLSISQUreIiIRpOQtIhJBSt4i\nIhGk5C0iEkFK3iIiEaTkLSISQUreIiIRpOQtIhJBSt4iIhGk5C0iEkFK3iIiEaTkLSISQUreIiIR\npOQtIhJBSt4iIhGk5C0iEkFK3iIiEZQ2eZvZhWb2DzN7x8y+306be7z9b5pZVfBhiohIog6Tt5kV\nAr8CLgROBa4ws1OS2kwETnLODQOuBe7PUqw5dufwfEeQmajFC9GLWfFmX9Rizl+86e68xwDrnHMb\nnHPNwBPApUltJgHzAJxzS4EyMysPPNKcWzIi3xFkJmrxQvRiVrzZF7WY8xdvuuQ9BNiUsL3Z+y5d\nm4ojD01ERNqTLnk7n8exTv6ciIh0Qo80+7cAlQnblcTurDtqU+F9l4LNySy8fLOL8x1BZqIWL0Qv\nZsWbfVGLOT/xpkvey4BhZnY8sBX438AVSW2eBqYBT5jZWKDOObcj+UDOueS7cxER6aQOk7dzrsXM\npgHPAoXAXOfcajO7zts/xzn3jJlNNLN1wAHg21mPWkSkmzPnNDwtIhI1gc6wNLOHzGyHma3ooE1o\nJvSki9fMzjOzD8zsde/PrbmOMSmeSjN7zszeNrOVZlbTTrtQ9LGfeEPYxyVmttTM3jCzVWZ2Rzvt\nwtLHaeMNWx97MRV6sSxqZ38o+jchnnbjzVv/OucC+wOMB6qAFe3snwg8433+DPBKkOfPQrznAU/n\nM8akeI4BRnuf+wJrgFPC2sc+4w1VH3sx9fb+twfwCnBuWPvYZ7xh7OPpwGOp4gpb//qINy/9G+id\nt3PuBWBfB01CNaHHR7xweBlk3jjntjvn3vA+7wdWA8cmNQtNH/uMF0LUxwDOuQbvY09iz3r2JjUJ\nTR97MaSLF0LUx2ZWQSxB/5bUcYWqf33ESwffZ02uF6aK2oQeB5zj/dPtGTM7Nd8BxXkVQFXA0qRd\noezjDuINXR+bWYGZvQHsAJ5zzq1KahKqPvYRb9j6+JfADKCtnf2h6l/Sx5uX/s3HqoJRmtDzGlDp\nnDsDuBdYmOd4ADCzvsAC4EbvjvawJknbee3jNPGGro+dc23OudHEEsZnzey8FM1C08c+4g1NH5vZ\nxcBO59worqwJAAABWElEQVTrdHy3Gor+9RlvXvo318k7gwk9+eecq4//k9Q5txgoMrOj8hmTmRUB\nTwKPOudS/SUJVR+nizeMfRznnPsA+CNwdtKuUPVxXHvxhqyPzwEmmdl7wOPA+Wb2u6Q2YerftPHm\nq39znbyfBq4E6GhCT1iYWbmZmfd5DLHSylTjibmKx4C5wCrn3N3tNAtNH/uJN4R9PNDMyrzPvYAv\nAa8nNQtTH6eNN0x97Jz7oXOu0jn3aeAbwBLn3JVJzULTv37izVf/ppthmREzexz4HDDQzDYBtwFF\nEM4JPeniBaqB682sBWgg9n9ePo0Dvgm8ZWbx/0B/CAyFUPZx2ngJXx8PBuaZWQGxm5tHnHN/tfBO\nTEsbL+Hr40QOIMT9m+yweMlT/2qSjohIBOk1aCIiEaTkLSISQUreIiIRpOQtIhJBSt4iIhGk5C0i\nEkFK3iIiEaTkLSISQf8fkX/ORm5c+7cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108995550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEPCAYAAABCyrPIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHwtJREFUeJzt3XvUXXV95/H3JyGSCOF5iFgI5MYQ0OHSEtsCii1hdCzE\nES+rRekMOniXSijTcaiOFsYZV7Eub9AOMoAW8LYsjghjAG1LAFkKFUkgXOQikIRLQAiBEKCQfOeP\nvQ/PyZNz2ec8e5+z99mf11rPynPO2Zdvfjk53/P7/n77txURmJmZTRt2AGZmVg5OCGZmBjghmJlZ\nygnBzMwAJwQzM0s5IZiZGVBgQpA0U9KNklZJukPSX7fZ7mxJ90haLWlJUfGYmVlnOxV14Ih4XtLR\nEbFF0k7ATyW9MSJ+2thG0jJgcUTsL+lw4FzgiKJiMjOz9gotGUXElvTXVwDTgScnbXIccFG67Y3A\nuKQ9i4zJzMxaKzQhSJomaRWwAbgmIu6YtMk+wLqmx+uBeUXGZGZmrRXdQ9gWEYeSfMj/oaSlLTbT\n5N2KjMnMzForbAyhWURskvQj4PeAlU0vPQTMb3o8L31uO5KcJMzM+hARk790d9y4kB9gD2A8/X0W\ncB3wpknbLANWpL8fAfy8zbGiuDiPuQoidvw59sqizjnFdj1z2DGUpQ3L2BZ+Xwz/x22xXVtEL9sX\nWTKaC/xzOoZwI3BFRPyTpI9I+kga6Qrg15LuBc4DTi4wnjbWnQ0funf75z54H6w9Z/CxVNW6s+HD\n923/nNvQrGqKnHZ6G/C6Fs+fN+nxx4uKIYuINSukg4ET/gpeczg8fDf8/LSINSuGGVeVJG34H38L\nPvMN2PosrLoe1p7jNjSrloGMIZRd8oHGNOAK4J8jKPMH2cphB9Dat34N3AHMi+DYAZ105YDOUwUr\nhx1AiawcdgBV5aUrJowBTwMLhx1IJxGxctgxtLEQWA1MkxgfxAlL3BYD57aY4LbonxPChHHgNkqe\nEEpsIfBg+uM2NKsgJ4QJY8CtwAJph2sjrLsFTCSEBUOOxcz64IQwYYzkw2wbsPuQY6ki9xDMKs4J\nYcI48BT+QOvXQmBt+uP2M6sgJ4QJY8AmXPLoWVpic8nIrOKcECY0EoK/4fZuD+C5CDbjHpZZZTkh\nTHDJqH+NchE4oZpVlhPChOaSkT/QetMYUAZ4BJgjMXOI8ZhZH5wQJowzUTJyDbw3jfEDIthKsmKt\n72thVjFOCBPGcMmoX809BHAbmlWSEwIgMYPkNp9bgEeBMYlZw42qUprHEMDjCGaV5ISQGAOeTpcE\n30ZyK0+XjbJzD8FsBDghJBrlogaPI/Tm5TGElK9FMKsgJ4REY4ZRg7/hZiSxC/BK4PGmp91+ZhXk\nhJBozDBq8AdadguBdRE03/faYwhmFeSEkJhcMnLJI7vJ5SJIEsK89KZDZlYR/g+bmFwy8jfc7CYP\nKBPB88BGYK+hRGRmfXFCSDSWrWhwySi7yVNOG5xUzSrGCSExuYewDthHYvqQ4qmSHXoIKSdVs4px\nQkhsN6gcwQvAE8DcoUVUHa3GEMDjMGaV44SQmDyoDP6Gm5V7CGYjwgkhMblkBP5A6ypd8mMvksXs\nJvMYglnFOCEkJl+HAC55ZLE3sCGCF1u85vYzqxgnhESrkpG/4XbXrlwE7mGZVY4TQsIlo/60m3IK\nSYKdJjE+wHjMbAqcEBLtSkZOCJ217SGkS1m4Dc0qpPYJQUK07iGsBRakr1tr7aacNngcwaxCap8Q\ngJnA1vTag5dF8BSwDdh9KFFVQ6cxBHAPwaxSnBBal4sa/IHWWacxBPDAvFmlOCG0nmHU4ITQRlpK\ny1IycvuZVYQTQucegu+c1t4ewHMRbO6wjccQzCrECcE9hH51KxeBS0ZmleKE0HqGUYMTQnvdBpQB\nHgHmSMwcQDxmNkVOCN0HlV3yaK3b+AERbCVZ52jeQCIysylxQuhcMnLJo70sPQRwL8usMpwQOpeM\nHgXGJGYNMJ6qyDKGAE6qZpXhhNChZBTBNmA9Lhu14h6C2YhxQuhcMgJPPW2n6xhCyuMwZhXhhNC5\nZAT+hrsDiV2AXYDHM2zukpFZRTghdJ5lBE4IrSwE1qYrmnbj9jOrCCeE7iUjlzx2lLVcBEkPYZ7k\n95pZ2fk/afcegkseO8o6oEwEzwMbSe69bGYl5oTgMYR+ZJ1y2uCkalYBtU4IaRljV+DpDputA/aR\nmD6YqCohcw8h5aRqVgG1TgjAbGBLusRCS+mNc54A5g4sqvLrZQwBPA5jVgmFJQRJ8yVdI+l2SWsk\nLW+xzVJJmyTdkv58uqh42ug2oNzgb7jbc8nIbATtVOCxXwROi4hVknYFbpb0k4i4c9J210bEcQXG\n0Um38YOGRkK4odhwyk9iBskA8foednsQeEsxEZlZXgrrIUTEoxGxKv19M3AnsHeLTYd5E/tuM4wa\n3EOYsA+wIYIXe9jH7WdWAQMZQ5C0CFgC3DjppQDeIGm1pBWSDhxEPE2yloy8fMWEXscPwGMIZpVQ\nZMkIgLRcdClwatpTaPZLYH5EbJF0LHAZcECb45zZ9HBlRKzMIbxeSkbLcjjfKOh1/ACSpDtNYjwi\nUwI2sz5IWgos7Xf/QhOCpBnA94FvRsRlk1+PiGeafr9S0v+WNCcinmyx7ZkFhOiSUe96nXJKBCG9\n3IZOCGYFSb8or2w8lnRGL/sXOctIwIXAHRHxlTbb7Jluh6TDALVKBgXqZZbRAmmo4x1l0U/JCFw2\nMiu9InsIRwL/CbhV0i3pc58i/VCIiPOAPwY+JuklYAvwngLjaWWc5BqDjiLYJLEN2B0YZMIqo4XA\nD/rYz70ss5IrLCFExE/p0gOJiL8D/q6oGDIYA36dcdvGB5oTQu9jCOBrEcxKr+5XKmctGYG/4ZKW\nzKZSMqp1+5mVXd0TQtZBZfDUU4A9gOcimDxbLAuPIZiVXN0TQtZpp+BvuNB/uQhcMjIrPScEl4x6\n0fOU0yaPAHMkZuYYj5nlqO4JoZeSkUse/Y8fkK4o+xAwL9eIzCw3dU8IvZSMXPKYWg8B3MsyK7Xa\nJoR01c5XAM9m3OVRYExiVnFRld5UxhDASdWs1LomBEl/LmlMiQvT+xb80SCCK9gY8HQEkWXjCLaR\nLPlc57KRewhmIyxLD+H9EbGJZD37OcCJwFmFRjUYvQwoN9R9HKHvMYRU3dvPrNSyJITG+j1vBS6J\niDUFxjNIvQwoN9S25CGxC7AL8PgUDlPb9jOrgiwJ4WZJPyZZ/vlqSbsB24oNayB6GVBuqHPJYyGw\nNmuJrY06t59Z6WVZy+gDwKHAfRHxrKRXAScVG9ZA9FsyOrqAWKpgquMHkPQQ5klMS8dkzKxEuiaE\niNgqaQNwoKSdSEpIU/mWWBb9lozqWgOf6vgBETwvsZHknswP5xKVmeWma0KQ9Hng3cAdwNaml64r\nKqgBccmoN1OdctrQGEdwQjArmSwlo3cCr4mIF4oOZsD6KRmtA/aWmJ5eeVsnC4GrczhOI6n+LIdj\nmVmOsgwq30dyAdeo6blkFMELJDfUmVtIROU25ZJRylNPzUoqSw/hOWCVpH8CGr2EiIjlxYU1EGNA\nP1NoGyWP9fmGU3p5loxek8NxzCxnWRLC5elPYyB5VAaV+ykZwUTJ44Z8wymvdJmPvcgnCT5IcpGj\nmZVMlllGfy9pZ+CA9Km7IuLFYsMaiH5mGUE9B5b3ATZEkMe/ex3bz6wSsswyWgpcxET9eIGk90XE\ntUUGNgD9zDKCpORxUM6xlF1e4wfgMQSz0spSMvoS8JaI+BWApAOA7wKvKzKwARin/5LRspxjKbu8\nxg8gafNpEuMRfbW/mRUkyyyjnRrJACAi7iZbIim7fnsIdSx55HGVMgDp0hd1bEOz0su6ltEFkpZK\nOlrSBcAvig6sSBJiaglhQXqMusizZAQuG5mVUpaE8DHgTmA5cApwe/pclc0CtqbXFfQkgk0ki/vt\nnntU5ZVnyQi86qlZKWWZZfQ88MX0Z1T02ztoaJQ8nswnnNLLrWSUcsnIrITaJgRJ/xARfyJpDTte\ndxAR8dvFhlaofq9BaGh8oN2STzjllZbGiigZVX1SgtnI6dRDODX9862wQ7286hem9XsNQkOdauB7\nAM9FsDnHY9ap/cwqo+0YQkQ0VqM8OSIeaP4BTh5IdMWZasmoTjXwvMcPoF7tZ1YZWQaVWy0zUPV5\n+HmVjOog7/EDgEeAORIzcz6umU1BpzGEj5H0BPaTdFvTS7Op/jo+eZSMnBD6FMFWiYeA+cA9eR7b\nzPrXaQzh28CVwFnA6UyMIzwTEU8UHVjB8igZ1aUGvoD8S0Yw0YZOCGYl0WkMYVM6ZvCeiHgQ2EIy\n/34XSVX/MOx32YqGR4ExiVk5xVNmRZSMoF69LLNK6DqGIOk4SfcA9wPXAg+Q9ByqbEo9hPQG8eup\nRy/BCcGsJrIMKv8v4PXA3RGxL/Am4MZCoyreVEtGUJ+pk3lfg9BQl/Yzq4wsCeHFiPgNME3S9Ii4\nBvi9guMq2lRLRlCDqZMSuwC7AI8XcPiRbz+zqsmyaulGSbOB64FvSXoMcr1IaRjy6iGM+gfaQmBt\nukJp3urQfmaVkqWH8HaSAeXTgKuAe4G3FRnUAEz1OgSoxwdaUeMHkPQQ5kmZ3oNmNgBZegi/BTwa\nEc8Bfy9pFrAnUOWpp1O9DgHqMfW0qPEDInheYiPJvZof7ra9mRUvy7ezS4GtTY+3pc9VmUtG2RSx\nbEUzjyOYlUiWhDA9Iv618SAiXgBmFBdSsdISxa7A01M81Dpgb4npU4+qtIosGUE9kqpZZWRJCL+R\n9PbGg/T33xQXUuFmA1situv19Cy9uc4TwN65RFVOTghmNZJlDOGjJLOL/jZ9vB44sbiQCpdHuaih\nMY6wLqfjlU1Ry1Y0rAVeU+DxzawHWe6Ydi9wuKRd08dVn3KaxzUIDY1vuFVf7G8HEjNIBnzXF3ia\nB2m9mq6ZDUGn1U5PjIhLJP0FTTfEkSSSO6Z9aRABFiDPHsIolzz2ATZE8GKB5xjl9jOrnE49hFem\nf86m+ndIa5Z3Qjg4p2OVTWFTTpt4+QqzEumUEPZL/7wjIr43iGAGJM+S0VqSW4yOoqKnnELy7zBN\nYjwit38TM+tTp1lGy9Ly0CcHFcyAuGSUTdEzjEiXxBjlNjSrlE4J4UpgI3CIpGcm/XSdwy9pvqRr\nJN0uaY2k5W22O1vSPZJWS1rS59+jF7knBOnlmweNksITQsoJwawk2paMIuITwCckXR4Rx/Vx7BeB\n0yJiVTpD6WZJP4mIOxsbSFoGLI6I/SUdDpwLHNHHuXoxTm7Lbhx8JBy3M9x/vfTUZlh3dsSaFZ32\nkA5eBvOXw+yZ8MzzWfbpd7+p7XPIYfDQ66TVa7PE14/kXEcdAvFF6f6Ty9sW5fy3GuS5yh7fIM9V\n9vi2369HETGQH+Ay4E2Tnvsa8O6mx3cBe7bYN/KLI86D+OjUj3PQMvjgPRAx8fPBe+CgZXnuM8hz\n9RvfoNqvCm0xiucqe3xui077Eb38v+w07fSGiDhS0mZ2nGUUEbFb1qQjaRGwhB1vrLMP21/UtR6Y\nB2zIeuw+5LHSKUn2PX/x9s+dvxhO/J/tl7M49H/0vk+/++W5z7JTgJx7Cf20X6cYB9UWZYlvkOcq\ne3yDPFfZ42u3XzadSkZHpn/u2s+BG9Jy0aXAqdH6orbJ9feWU1wlndn0cGVErOwzpDxWOiXpwrWy\n10Lgw61fm9tmimWnffrdL899di3g3tH9tB8Mvy3KEt8gz1X2+AZ5rrLH9/1Xwb2vhTPbh9JJty4E\nyfTTmenvRwPLgfEs3Q+SRfCuBv68zetfA97T9HgQJaOfQbxh6sc55qrtu3KNn2OvzHOfQZ6r3/gG\n1X5VaItRPFfZ43NbdNqP6On/Zff/uKwm6UksBu4GvgCsyLCfgIuBL3fYZlnjWCSDyT9vs11Pf6nO\nccWdEAdO/Tit6nsfuLf3mmDnfQZ5rn7jG1T7VaEtRvFcZY/PbdFpP6KX/5dKDtCepFsiYomk/wY8\nFxHnNJ7rst8bgeuAW+HlMtCnSK9MjYjz0u3+FjgGeBY4KSJ+2eJYERG5TO2UeBj4/QgemvqxDl4G\nC05JSiqbn4O152SbNdDbPoM8V7/x9WNU22IUz1X2+AZ5rrLHt/1+Vx7Ty2dnloRwI/BVkg/ztwEP\nALdFxMCWbMg5ITwL7BlR+ftCm5l11OtnZ5b7IbyfpJzzuYi4H1gEXNJfeMOVruC5M0lvxMzMmnTt\nIWy3sTQHmBcRtxYXUsvz5tJDkNgD+FUEr8ohLDOzUsu9hyDpWkm7pcngZuACSV+eSpBDlOeyFWZm\nIyVLyWgsIp4G3gVcHBGHAW8uNqzC5LnSqZnZSMmSEKZLmgscD/wofS57nalc3EMwM2sjS0L4LMnF\nZfdFxE2S9gPuKTaswjghmJm10dOg8rDkOKh8EnBUBP956lGZmZVbr5+dne6Y1jjgLOADwIFAY02b\niIj39xfiULmHYGbWRpaS0SXAniRXE68kWY20qhd1eVDZzKyNLAlhcUR8BtgcEReRrD90eLFhFcY9\nBDOzNrIkhH9N/9wk6RCSb9mvLi6kQjkhmJm10XUMATg/vSjt08DlwK7AZwqNqjguGZmZtdE1IUTE\n+emv1wL7FhtO4dxDMDNro9MtNP+ixdNBcp+DiIgvFRZVcZwQzMza6NRDmE11r0huxyUjM7M26nZh\n2uPAQRE8lkNYZmalVsRqpxdJGm96vLukr/cb4LBICJeMzMzayjLt9Hci4uUyS0RsBF5XXEiFmQVs\njeCFYQdiZlZGWRKC0mmnjQdzgOnFhVQY9w7MzDrIch3CF4GfSfoeyQyjPwE+V2hUxRjHCcHMrK0s\n1yFcLOlm4N+RzDp6Z0TcUXhk+RvDM4zMzNrKstrpfiT3Qrhd0tHAmyU93DyuUBEuGZmZdZBlDOH/\nAi9JWgycB8wHvl1oVMXwNQhmZh1kSQjbIuIlknsqnxMRnwDmFhtWIdxDMDPrINNqp5L+FHgv8P8k\nCZhRbFiFcEIwM+sgS0J4P3AE8LmIuB9YRHLTnKpxycjMrIMss4xuB5bDy9cgzI6IzxcdWAHGgEeH\nHYSZWVllWbriWkm7pcngZuACSV8uPrTcuWRkZtZBlpLRWEQ8TTKofHFEHAa8udiwCuGSkZlZB1kS\nwnRJc4HjgR+lz5V/idQduYdgZtZBloTwWeBqkovTbkovVLun2LAK4aUrzMw6qM39ECQeBI6K4IF8\nojIzK7dePzs73ULz9Ij4vKRzmLh1ZkNExPIpxDkMLhmZmXXQadppYwG7m1u8Vv5uRROJaSS3BH16\n2LGYmZVVLUpGEmPAugh2yzEsM7NSy7NkdAU7looaIiKO6yO+YXG5yMysi04loyOA9cB3gBvT5xrJ\nofzdiu35GgQzsy46JYS5wL8HTkh/fgR8J13KomrcQzAz66LtdQgR8VJEXBkR7yXpLdwLXCvp4wOL\nLj9OCGZmXXRc3E7STOCtwHtIVjn9KvCD4sPKnUtGZmZddBpUvgQ4CFgBfDYibhtYVPlzD8HMrIu2\n004lbQOebbNfRMTApnDmMO30vwO7RvDJHMMyMyu13KadRkSWdY6qYgx4YthBmJmV2Sh96HfikpGZ\nWRd1SQhe6dTMrIu6JIQxPMvIzKyjOiUE9xDMzDooNCFI+rqkDZJaTlmVtFTSJkm3pD+fLigUl4zM\nzLroeGFaDr4BnANc3GGbawewUJ5LRmZmXRTaQ4iI64GNXTab0p3QMnLJyMysi2GPIQTwBkmrJa2Q\ndGDeJ5CYAexM+4vszMyM4ktG3fwSmB8RWyQdC1wGHNBqQ0lnNj1cGRErM55jDNgUUbklu83MeiJp\nKbC07/2LvmOapEXAFRFxSIZt7wd+NyKenPR830tXSCwGfhzBv+lnfzOzqur1s3OoJSNJe0pS+vth\nJAnqyS679coDymZmGRRaMpL0HeAoYA9J64AzgBkAEXEe8MfAxyS9BGwhWWY7bx5QNjPLoPCSUR6m\nWDJ6F/DeCN6Rc1hmZqVWqZLRgLhkZGaWQV0SgktGZmZd1CEheNkKM7MM6pAQXDIyM8ugLgnBPQQz\nsy7qkBBcMjIzy6AOCcElIzOzDOqQENxDMDPLoA4JwT0EM7MM6pIQ3EMwM+tipBOChHDJyMwsk5FO\nCMAs4KUIXhh2IGZmZTfqCcHlIjOzjEY9IbhcZGaW0agnBM8wMjPLqA4JwT0EM7MMRj0huGRkZpbR\nqCcEl4zMzDIa9YTgHoKZWUajnhA8hmBmllEdEoJLRmZmGYx6QnDJyMwso1FPCO4hmJllVIeE4B6C\nmVkGo54QXDIyM8to1BOCS0ZmZhnVISG4h2BmlsHIJgSJacBs4Olhx2JmVgUjmxBIksGzEWwddiBm\nZlUwygnBA8pmZj0Y5YTg8QMzsx6MekLwDCMzs4xGOSG4ZGRm1oNRTgguGZmZ9WDUE4JLRmZmGY1y\nQnDJyMysB6OcENxDMDPrwagnBPcQzMwyGuWE4JKRmVkPRjkhuGRkZtaDUU4I7iGYmfVglBOCxxDM\nzHow6gnBJSMzs4xGOSG4ZGRm1oORTAgSM4BXAM8OOxYzs6oYyYRAOn4QQQw7EDOzqig0IUj6uqQN\nkm7rsM3Zku6RtFrSkpxO7XKRmVmPiu4hfAM4pt2LkpYBiyNif+DDwLk5nXdkZxhJWjrsGMrCbTHB\nbTHBbdG/QhNCRFwPbOywyXHARem2NwLjkvacyjmlg5fBCefC6YukY69KHo+UpcMOoESWDjuAElk6\n7ABKZOmwA6iqnYZ8/n2AdU2P1wPzgA39HCz58H/9V+H8xelTfwQf2k86mIg1K6YYq5nZSCvDoLIm\nPZ7CQPD85U3JIHX+YlhwSv/HNDOrB0UUOxFH0iLgiog4pMVrXwNWRsR308d3AUdFxIZJ23m2kJlZ\nHyJi8pfutoZdMroc+DjwXUlHAE9NTgbQ21/IzMz6U2hCkPQd4ChgD0nrgDOAGQARcV5ErJC0TNK9\nJBeRnVRkPGZm1l7hJSMzM6uGMgwqtyXpGEl3pReunT7seIZJ0gOSbpV0i6Sbhh3PILW6wFHSHEk/\nkXS3pB9LGh9mjIPSpi3OlLQ+fW/cIqnttT+jRNJ8SddIul3SGknL0+dr997o0BY9vTdK20OQNB34\nFfBm4CHgX4ATIuLOoQY2JJLuB343Ip4cdiyDJukPgM3AxY3JCZL+BvhNRPxN+mVh94j4y2HGOQht\n2uIM4JmI+NJQgxswSXsBe0XEKkm7AjcD7yApPdfqvdGhLY6nh/dGmXsIhwH3RsQDEfEi8F3g7UOO\nadhqObje5gLHly9qTP98x0CDGpIOF3vW7r0REY9GxKr0983AnSTXNtXuvdGhLaCH90aZE0Kri9b2\nabNtHQTwj5J+IelDww6mBPZsmpG2AZjSFe4j4JR0PbAL61AimSyd3r4EuJGavzea2uLn6VOZ3xtl\nTgjlrGUNz5ERsQQ4FviztHRgQCR1zzq/X84F9gUOBR4BvjjccAYrLZF8Hzg1Ip5pfq1u7420LS4l\naYvN9PjeKHNCeAiY3/R4PkkvoZYi4pH0z8eBH5CU1OpsQ1o3RdJc4LEhxzM0EfFYpIALqNF7Q9IM\nkmRwSURclj5dy/dGU1t8s9EWvb43ypwQfgHsL2mRpFcA7ya5kK12JL1S0uz0912AtwBtlxSvicuB\n96W/vw+4rMO2Iy390Gt4JzV5b0gScCFwR0R8peml2r032rVFr++N0s4yApB0LPAVYDpwYUT89ZBD\nGgpJ+5L0CiC5mPBbdWqL5gscSWrCfwX8EPgesAB4ADg+Ikb+Htot2uIMktU9DyUpjdwPfKTVFf+j\nRtIbgeuAW5koC30SuImavTfatMWngBPo4b1R6oRgZmaDU+aSkZmZDZATgpmZAU4IZmaWckIwMzPA\nCcHMzFJOCGZmBjghWA1I2pz+uVDSCTkf+1OTHt+Q5/HNBskJweqgcbHNvsCf9rKjpG53FfzkdieK\nOLKX45uViROC1clZwB+kNwo5VdI0SV+QdFO6GuSHASQtlXS9pB8Ca9LnLktXml3TWG1W0lnArPR4\nl6TPNXojSo99W3pjo+Objr1S0j9IulPSNxvBSTorvcHJaklfGGjLmFHwPZXNSuZ04L9GxNsA0gTw\nVEQcJmln4KeSfpxuuwQ4KCIeTB+fFBEbJc0CbpJ0aUT8paQ/S1ehbWj0Rt4F/A7w28CrgX+RdF36\n2qHAgSSrT94g6UjgLuAdEfHaNLbdCvj7m3XkHoLVyeQbhbwFeK+kW0jWjp8DLE5fu6kpGQCcKmkV\n8DOSlXf373KuNwLfTheafAy4Fvh9koRxU0Q8nK5AuQpYCDwFPJ+uWf9O4Lm+/5ZmfXJCsLr7eEQs\nSX/2i4h/TJ9/trGBpKXAm4AjIuJQ4BZgZpfjBjsmoEbv4YWm57YCMyJiK8nSxJcC/wG4qp+/jNlU\nOCFYnTwDzG56fDVwcmPgWNIBkl7ZYr/dgI0R8byk1wJHNL32YpuB5+uBd6fjFK8G/pBkFc6WtzNM\nlzUfj4grgf9CUm4yGyiPIVgdNL6Zrwa2pqWfbwBnA4uAX6bryT9Gsmb85LtsXQV8VNIdwK9IykYN\n/we4VdLNEXFiY7+I+IGk16fnDOATEfGYpH/LjnfwCpJE9UNJM0mSxmm5/M3NeuDlr83MDHDJyMzM\nUk4IZmYGOCGYmVnKCcHMzAAnBDMzSzkhmJkZ4IRgZmYpJwQzMwPg/wMEND6+4W+FpwAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108a38a58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# versicolor and virginica\n",
    "y2 = df.iloc[50:150, 4].values\n",
    "y2 = np.where(y2 == 'Iris-virginica', -1, 1)\n",
    "\n",
    "# sepal width and petal width\n",
    "X2 = df.iloc[50:150, [1,3]].values\n",
    "\n",
    "ppn = Perceptron(epochs=25, eta=0.01)\n",
    "ppn.train(X2, y2)\n",
    "\n",
    "plot_decision_regions(X2, y2, clf=ppn)\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(1, len(ppn.errors_)+1), ppn.errors_, marker='o')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Misclassifications')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of misclassifications: 43 of 100\n"
     ]
    }
   ],
   "source": [
    "print('Total number of misclassifications: %d of 100' % (y2 != ppn.predict(X2)).sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even at a lower training rate, the perceptron failed to find a good decision boundary since one or more samples will always be misclassified in every epoch so that the learning rule never stops updating the weights.\n",
    "\n",
    "It may seem paradoxical in this context that another shortcoming of the perceptron algorithm is that it stops updating the weights as soon as all samples are classified correctly. Our intuition tells us that a decision boundary with a large margin between the classes (as indicated by the dashed line in the figure below) likely has a better generalization error than the decision boundary of the perceptron. But large-margin classifiers such as Support Vector Machines are a topic for another time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_margin.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adaptive Linear Neurons and the Delta Rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The perceptron surely was very popular at the time of its discovery, however, it only took a few years until Bernard Widrow and his doctoral student Tedd Hoff proposed the idea of the Adaptive Linear Neuron (adaline) [[3](#References)].\n",
    "\n",
    "In contrast to the perceptron rule, the delta rule of the adaline (also known as Widrow-Hoff\" rule or Adaline rule) updates the weights based on a linear activation function rather than a unit step function; here, this linear activation function $g(\\mathbf{z})$ is just the identity function of the net input $g(\\mathbf{w}^T\\mathbf{x}) = \\mathbf{w}^T\\mathbf{x}$. In the next section, we will see why this linear activation is an improvement over the perceptron update and where the name \"delta rule\" comes from."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_vs_adaline.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Being a continuous function, one of the biggest advantages of the linear activation function over the unit step function is that it is differentiable. This property allows us to define a cost function $J(\\mathbf{w})$ that we can minimize in order to update our weights. In the case of the linear activation function, we can define the cost function $J(\\mathbf{w})$ as the *sum of squared errors* (SSE), which is similar to the cost function that is minimized in ordinary least squares (OLS) linear regression.\n",
    "\n",
    "$$J(\\mathbf{w})  = \\frac{1}{2} \\sum_{i} (\\text{target}^{(i)} - \\text{output}^{(i)})^2 \\quad \\quad \\text{output}^{(i)} \\in \\mathbb{R}$$\n",
    "\n",
    "(The fraction $\\frac{1}{2}$ is just used for convenience to derive the gradient as we will see in the next paragraphs.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to minimize the SSE cost function, we will use gradient descent, a simple yet useful optimization algorithm that is often used in machine learning to find the local minimum of linear systems."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we get to the fun part (calculus), let us consider a convex cost function for one single weight. As illustrated in the figure below, we can describe the principle behind gradient descent as \"climbing down a hill\" until a local or global minimum is reached. At each step, we take a step into the opposite direction of the gradient, and the step size is determined by the value of the learning rate as well as the slope of the gradient."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_gradient_descent_1.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, as promised, onto the fun part -- deriving the Adaline learning rule.\n",
    "As mentioned above, each update is updated by taking a step into the opposite direction of the gradient $\\Delta \\mathbf{w} = - \\eta \\nabla J(\\mathbf{w})$, thus, we have to compute the partial derivative of the cost function for each weight in the weight vector:  $\\Delta w_j = - \\eta \\frac{\\partial J}{\\partial w_j}$. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The partial derivative of the SSE cost function for a particular weight can be calculated as follows:\n",
    "\n",
    "$$\\begin{equation}\n",
    " \\frac{\\partial J}{\\partial w_j} = \\frac{\\partial }{\\partial w_j} \\frac{1}{2} \\sum_i  (t^{(i)} - o^{(i)})^2 \\\\\n",
    "= \\frac{1}{2} \\sum_i \\frac{\\partial}{\\partial w_j} (t^{(i)} - o^{(i)})^2 \\\\\n",
    "= \\frac{1}{2} \\sum_i 2 (t^{(i)} - o^{(i)}) \\frac{\\partial}{\\partial w_j} (t^{(i)} - o^{(i)}) \\\\\n",
    "=  \\sum_i (t^{(i)} - o^{(i)}) \\frac{\\partial}{\\partial w_j} \\bigg(t^{(i)} - \\sum_j w_j x^{(i)}_{j}\\bigg) \\\\\n",
    "= \\sum_i  (t^{(i)} - o^{(i)})(-x^{(i)}_{j}) \n",
    "\\end{equation}$$\n",
    "\n",
    "(t = target, o = output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And if we plug the results back into the learning rule, we get\n",
    "\n",
    "$\\Delta w_j = - \\eta \\frac{\\partial J}{\\partial w_j} = - \\eta \\sum_i  (t^{(i)} - o^{(i)})(- x^{(i)}_{j}) = \\eta \\sum_i (t^{(i)} - o^{(i)})x^{(i)}_{j}$,\n",
    "\n",
    "Eventually, we can apply a simultaneous weight update similar to the perceptron rule: \n",
    "\n",
    "$\\mathbf{w} := \\mathbf{w} + \\Delta \\mathbf{w}$.\n",
    "\n",
    "**Although, the learning rule above looks identical to the perceptron rule, we shall note the two main differences:**\n",
    "\n",
    "1. Here, the output \"o\" is a real number and not a class label as in the perceptron learning rule.\n",
    "2. The weight update is calculated based on all samples in the training set (instead of updating the weights incrementally after each sample), which is why this approach is also called \"batch\" gradient descent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Gradient Descent Rule in Action"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, it's time to implement the gradient descent rule in Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class AdalineGD(object):\n",
    "    \n",
    "    def __init__(self, eta=0.01, epochs=50): \n",
    "        self.eta = eta\n",
    "        self.epochs = epochs\n",
    "\n",
    "    def train(self, X, y):\n",
    "\n",
    "        self.w_ = np.zeros(1 + X.shape[1])\n",
    "        self.cost_ = []\n",
    "\n",
    "        for i in range(self.epochs):\n",
    "            output = self.net_input(X)\n",
    "            errors = (y - output)\n",
    "            self.w_[1:] += self.eta * X.T.dot(errors)\n",
    "            self.w_[0] += self.eta * errors.sum()\n",
    "            cost = (errors**2).sum() / 2.0\n",
    "            self.cost_.append(cost)\n",
    "        return self\n",
    "\n",
    "    def net_input(self, X):\n",
    "        return np.dot(X, self.w_[1:]) + self.w_[0]\n",
    "\n",
    "    def activation(self, X):\n",
    "        return self.net_input(X)\n",
    "\n",
    "    def predict(self, X):\n",
    "        return np.where(self.activation(X) >= 0.0, 1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In practice, it often requires some experimentation to find a good learning rate for optimal convergence, thus, we will start by plotting the cost for two different learning rates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEZCAYAAACervI0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHVWB/vHvy5pAwi6LmLCICxL2RRCEICgSHFQUFFTQ\nkUVlCANuoKPgMoK7GGcQURxcQBEMuACC/Iiisih7QEQQDAgEZJEEEmV5f3+caug0nfTtpW7d7vt+\nnqef9K1bdevtTnJO1TmnzpFtIiKi+yzVdICIiGhGKoCIiC6VCiAiokulAoiI6FKpACIiulQqgIiI\nLpUKIPol6f8kfarFfWdJenf1/dsk/aLedJ1L0isl3dJ0johWpALoMlVh/ZCk5QbY1dVXK57Z1/b3\nbe8xnIxDJelOSbs1ce4eti+z/dImM/SQNFXSXcP8jN0k3SLpMUn/T9LkJey7mqSZkuZXfxf793pv\nWUlnS7pD0tOSdhlOrhgZqQC6iKT1ge2A+4G9Wzmkzjw1GEylNSSSOuL/jCo1n2MN4Bzgo8CqwB+A\nHy7hkP8BFgJrAm8DTpb0sl7v/xp4O3AfNf89RWs64h9ztM2BwC+B7wIH9X5D0paSrpH0qKQfAON6\nvbeqpJ9Jur+6e/ippHX7O4Gkd0q6rNfrpyUdJulWSQ9L+lqf/f9d0s3V5164pCvMoarKymMk3Sbp\n75J+KGnVXu//SNK9kh6R9KvehVbVFHaypPMlzQd2ra5u3y/p+uqYH0havtp/kavuJe1bvf8hSfdI\nulvSwdXva8PF/ByzJH1a0m+Bx4ANJb2r+v09Kul2SYdW+64IXAA8X9K86v21B/pd9LEPMNv2Obb/\nBRwPbC7pxf1kW7Ha/2O2H7f9W+A84B0Atp+w/dVq+1MD/qVFW6QC6C4HUq7gzgL2kLQmQNUcdC5w\nOuVK70fAm3j2Kk3At4DJ1dcCYJGCfAB7AdsAmwH7SdqjOu/rgWOBNwJrAJcBZw79x1us6ZQ7np2B\ndYCHKVerPX4ObAQ8D7gG+H6f4/cHPmV7AvAbyu9lX2APYIPq53rnYs692H0lvRY4CtgNeBEwlYGv\njN8OHAxMAP4KzAX2sr0S8C7gy5K2tP0Y8FrgHtsTba9k+74Wfhe9bQJc/8wPYj8O3AZM6WffFwNP\n2r6t17brq8+IDpUKoEtI2glYF/iJ7T8DNwMHVG9vDyxj+yTbT9k+B/h9z7G2H7I90/ZC2/OBzwCD\nacM90fajtu8CLgU2r7a/BzjB9p9sPw2cAGwhadJwftZ+HAb8l+17bD8BfAJ4c09zju3/s/1Yr/c2\nlzSx1/Hn2r682vef1bav2r7P9sPAT4EtlnD+xe27H3Ca7T/aXgAcx5Kb3Qz8X7X/07aftH2+7Tuq\nbL8GLgJeWe3f32ct8XfRx4rAo322PUqpfPqa0M++84CJ/ewbHSIVQPc4CLjI9rzq9Y94thno+cDf\n+uz/V6oCRNIKkk6pmjP+AfwKWHkQbdD39fr+cZ4tQNYDTqqahh4GHqy2P6d5SdLXq6aMeZKOafG8\nPdYHZvY6z83Ak8BakpaWdGLVJPIP4I7qmDWqPw3015Ha+2daQP+F4uL2XbH6fp0+n313Cz/LIlkk\n7SnpCkkPVj/bNGD1JRy/Pov5XfSz73xgpT7bVqYU7MPZNzrEMk0HiPpJGk+52lxK0r3V5uUphfhm\nwD08t9Bdj3K7D/B+yi3+drbvl7QFpalEDK8zbw6laWXAZh/b76HcMQz1PO/quYrvTdI7KE0iu9n+\nq6RVgIdoTwf4vUDvu51W7nye+X1XfQnnUJqFzrP9lKSZPJu9v7+bxf4u+nETvfqKqnb+F1bb+7oV\nWEbSRr2agTYHZrdwnmhI7gC6wxsoV3kbU/5Tbl59/xtKv8DlwJOSpqsM19sH2LbX8RMoV67/kLQa\npaliqMSzBdTXgY/0dLpKWlnSvsP4bIDlJI3r9bVMdZ7P9HQwS3qepJ5RUBOAfwIPVQXcZ/rJO9J6\nPvMs4F2SXippBeBjgzgWYLnq6+/A05L2BF7T6/25wOqSel+ZL+l30ddMYIqkfSSNo/y9X2f71r47\nVn0OPwY+Wd0x7gT8G2XAAdW5lq8+B6D399GQVADd4UBKW/Pdtu+vvuZSOnIPoIzK2IfSOfkg5W7h\nnF7HfwUYTylofkcZXbK4K/++QzH77tf7mYFzgc8CP6iaX26kdJYOx/mUZqaer48DJwE/AS6S9Cil\nwtuu2v87lOauv1GuVi/vJ/9AdzkD/cz97mv7QuCrlH6RW6tzQ6mQlnQ81fHzKJ26Z1HuWvanjLzp\nef8WSqf6X1RGWa3Nkn8Xi57I/jtlMMB/V5+/DfDWnvclfUTS+b0OeR/l38n9wPeA99j+Y6/3/0T5\nO3k+8AvgMdUw6itap7oWhKlq919RmhqWo9yiHltdQf6Q0sRwJ7Cf7UdqCRExikjamFIJLld1ikfU\nqrY7ANsLgV1tb0EZ+rZrdVt4DHCx7RcDl1SvI7qSpDdWTSOrUu6GfpLCP9ql1iagatwwlDuApSlj\njvemjDen+vMNdWaI6HCHUtrqbwOeAN7bbJzoJrU1AcEzj81fQxk5cLLtD0l62Paq1fsCHup5HRER\n7VPrMNDqVnYLSSsDv5C0a5/3LSlzgkRENKAtzwHY/oeknwNbA3MlrW37PknrUEYMLCKVQkTE0Nhu\nfeiy7Vq+KE9SrlJ9P54yE+BuwOeAD1fbj6FME9D3WNeVa5g/0/FNZ0imZOrGXMnUciYPZv867wDW\nAU6v+gGWAr5r+xJJ1wJnqSwgcidlzHlERLRZbRWA7RuBrfrZ/hCwe13njYiI1uRJ4MGZ1XSAfsxq\nOkA/ZjUdoB+zmg7Qj1lNB1iMWU0H6MespgP0Y1bTAYar1mGgQyXJHkxHRkREF5OmTINJ0+HCPQZT\ndmY20IiIUawU/jucBKduNNi5C9MEFBExqk2aXgr/wUsFEBExqq065FXX0gQUETEKSQh4G0zeZqif\nkTuAiIhRRmIDyrocH4B/fRgOuW2gY/qTCiAiYpSQWFriKOD3lGGo29pf+gpcfiRMu3DQn5dhoBER\nnU9ic+CbwHzgUJs/P3efwZWduQOIiOhgEuMlTgAupqzp/Kr+Cv+hSCdwRESHktgV+AZwLbCZzX0j\n+fmpACIiOozEqsAXgFcDh9v8tI7zpAkoIqJDSEhiP+AmYAEwpa7CH3IHEBHRESQmAf9DWUL3zTa/\nq/ucuQOIiGiQxFISh1Pa+f8AbNWOwh9yBxAR0RiJTYBTAQM729zczvPnDiAios0klpc4nvIw13eB\nV7a78IfcAUREtJXEjpSr/luBLW3ubipLKoCIiDaQWBk4AXg9MB34sU2jUzGkCSgiomYSrwdmUy66\np9ic03ThD7kDiIiojcQ6wAxgM+DtNr9qONIicgcQETHCqqGdhwDXA3+iTOPQUYU/5A4gImJESbyE\nMn/PeGB3mxsajrRYuQOIiBgBEstJfBT4LTAT2KGTC3/IHUBExLBJbEeZq/9uYGubvzYcqSWpACIi\nBkGaMg0mTYeJ4+DxJ+CAx+CA7YGjgTM7YXRPq7IiWEREi0rhv8NJcOpGz279wDz406H2T3/QXLIi\nK4JFRNRm0vRFC3+AL0yEpw5qJs/w1FYBSJok6VJJN0maLWl6tf14SXdLurb6em1dGSIiRoqEYL31\n+n93wvj2phkZdfYBPAEcZfs6SROAqyVdTJn17ku2v1TjuSMiRozEhsApMOH5/e8xf0FbA42Q2u4A\nbN9n+7rq+/nAH4F1q7fTvh8RHU9iGYkPAFcBF8Glb4dDblt0r4Nvhzkzmsg3XG0ZBSRpfWBL4Apg\nR+AISQdSFj94v+1H2pEjIqJVEltShnY+AmxvcxtcjTTlKZh2RGn2mb8A5sywZ5/fcNwhqX0UUNX8\nMwv4tO1zJa0JPFC9/SlgHdvv7nNMRgFFRCMkVgCOA94FfAg4fbQM7Rxs2VnrHYCkZYFzgO/ZPhfA\n9v293v8m9L/gsaTje72cZXtWfUkjIkBiN+AU4PfApjZzG460RJKmAlOHfHxddwCSBJwOPGj7qF7b\n17F9b/X9UcC2tg/oc2zuACKibSRWB74AvAp4n83PG440JJ10B7Aj8HbgBknXVts+AuwvaQvKaKA7\ngMNqzBARsVhlaCdvAb4MnEWZq39es6naJ08CR0RXkpgMnAxMBg6xuaLhSMOWJ4EjIpZAYmmJI4Br\ngMspk7eN+sJ/KDIZXER0DYkplKGd/wJ2srml4UiNyh1ARIx5EuMkPgVcCnwbmNrthT/kDiAixjiJ\nnSkrdN0EbG5zT8OROkYqgIgYkyRWAT4L7AUcYTOz4UgdJ01AETHmSOwDzKYMN5+Swr9/uQOIiDFD\n4vnA14CXAfvbXNZwpI6WO4CIGPUklpI4DLiecuW/RQr/geUOICJGNYmXUjp5lwV2tZndcKRRI3cA\nETEqSSwn8THgN5RpHHZK4T84uQOIiFFHYnvKA113AlvZzGk20eiUCiAiOpY0ZVpZiH3iOJi3EP51\nKlyyC7Av8J/AWaNlrv5OlMngIqIjlcJ/h5Pg1I2e3Xrsk7DVpbDvW20eai5dZ6plOmhJGwPrA08D\nf7Xd9Y9QR0TdJk1ftPAHOGEZmPaUvW8K/xGw2ApA0gbAUcA04G/APZTF3NeR9ALgZ8CXbd/ZhpwR\n0XUmjut/+4Tx7c0xdi3pDuCzwKmURduf6P1GtdTjrsDngP3qixcR3UjihbDulP7fnb+gvWnGrsUO\nA7W9H3AJsG0/7z1h+6Jqn4iIESGxjMSHgCth45lw6O2L7nHw7TBnRiPhxqABO4ElXWd7izbl6Tln\nOoEjuozE1pShnQ8A77H5S+kInnxEafaZvwDmzLBnn99w1I412LKzlQrgC8AVwDlu05ChVAAR3UNi\nBeATwIHAB4HvZmjn0NRRAcwHVgCeAhZWm217pSGnHChUKoCIriDxauAU4HfAUTYPNBxpVBvxYaC2\nJwwvUkTEoiRWB74E7AK81+aChiN1pVafA3g9sDNlbu1f2f5prakiYkySELA/pfA/kzJX//xmU3Wv\nASsASSdSRgJ9n/IcwHRJr7B9bN3hImLskFgP+DqwLrC3zVUNR+p6rfQB3AhsYfup6vXSwHW2N60t\nVPoAIsYMiaWBI4D/Ar4IfMHmiSUfFUNRx1QQBlYBHqxer1Jti4hYIonNKEM7HwdeYXNrw5Gil1Yq\ngBOAayRdSmkC2gU4ptZUETGqSYwDPgYcAnwEOM3m6WZTRV9LrAAkLUWZAG4HSj+AgWNs39uGbBEx\nCknsQplG5npgc5uUFx2qlT6Aq21v3aY8PedMH0DEKCOxKmV+sNcC/2FzXsORus5gy85WloS8WNIH\nJE2StFrP1zAyRsQYIiGJN1MWY/8XsEkK/9GhlTuAO3lup69tbzjAcZOA7wBrVsd/w/ZXq8rjh8B6\nlOXc9rP9SJ9jcwcQMQpIrAv8D/Bi4BCb3zYcqauN6B1A1QfwYdsb9PlaYuFfeQI4yvYmwPbA4dXC\nMscAF9t+MWW20XQoR4wyEktJvBe4rvraMoX/6NO2PgBJ5wJfq752sT1X0trALNsv7bNv7gAiOpTE\nxpRO3qUoV/03NRwpKh3ZByBpfWBL4EpgLdtzq7fmAmsN5rMiohkSy0scB/waOAPYKYX/6NbKcwBv\npbThH95n+watnEDSBOAc4Ejb86RnKyfbltTvLYik43u9nGV7Vivni4iRJ/EKylX/7ZTmnrsbjhSA\npKnA1CEfX+cU/9XSkT8DLrD9lWrbLcBU2/dJWge4NE1AEZ1JYiXgM8A+wJHA2Zmrv3ON+FQQklYE\njgYm2z5E0ouAl9j+2QDHCfgWcHNP4V/5CXAQZc3hg4BzWw0bEfUpq29Nml4WY5+3EHa6Ej7678Av\nKEM7H246Y4ysVjqBzwKuBg60vUlVIfzO9uYDHLcTpa3wBp4dRnoscBVwFjCZDAON6Ail8N/hJDh1\no2e3HvMELPNx+9MnNpcsBqOOFcGutr21pGttb1ltu36gCmA4UgFEtJe054VwwR7PfWfahfb5e7Y/\nUQxFHaOA/ilpfK8TvBD451DCRUSnWmPV/rdPGN//9hgLWhkFdDxwIfACSWcAOwLvrDFTRLSJxLLA\n++EFW/a/x/wFbQ0UbTXgHYDti4A3Ae+ijP3d2valdQeLiHpJbAv8HpgKdx0Kh9y26B4H3w5zZjQQ\nLdpkUMNAJX3C9nE15uk5T/oAImoisSLwKeAA4P3AGTYuHcGTjyjNPvMXwJwZ9uzzm00bgzHincB9\nPvyZjuA6pQKIqIfEHpR1eX8DHG3zQMORYgTVsSTkIp8/yP0jogNIPA/4MqUP7z02v2g4UnSAVkYB\n9bZVLSkiohbVXP1vB26kzL01JYV/9FjsHYCk3p0/prr6r+byse3p9UaLiOGQ2AA4GVgbeJ3NHxqO\nFB1mSXcAV1dfy1Ou/G8F/gxsASxXf7SIGAqJZSSOpozwuRTYNoV/9KeVJ4GvBHay/UT1elngN7Zf\nXluodAJHDInE5sA3gXnAoTa3DXBIjCF1PAm8CrBSr9cTq20R0SEkxkucAFxMafbZLYV/DKSVUUAn\nAtdImlW93oXydHBEdACJVwGnANcAm9nc13CkGCVaeg6gmrd/u+rllbZr/QeWJqCIgUmsBnweeDVw\nuM1PG44UDRvxJqBqYfjdgc1tnwcsJ2m7AQ6LiJpUQzv3A2YDj1OGdqbwj0FrpRP468DTwK62N67W\nA77I9ja1hcodQES/JCYB/wtsCBxsc3nDkaKD1NEJ/HLb7wMWAth+CFh2iPkiYggklpI4nNLO/3vK\nurwp/GNYWukE/pekpXteSHoe5Y4gItpAYhPKguxPAzvb/LHhSDFGtHIHMAOYCawp6TPAb4ETak0V\nEUgsL/EJYBbwHVL4xwhbYh9A1QG8A/AQsFu1+RLbtf4jTB9AdDuJnShX/X+ijPD5W8ORYhSoY03g\n62xvMexkg5AKILqVxMqUZ2/2BqYDP7Zpfc726Gp1dAL/UtKbVc0CFxH1kHgDcBOwNLCJzTkp/KNO\nrdwBzAdWAJ6iGglEmQ10pcUfNcxQuQOIMaysvDVpOkwcB/MWwsTvw1lvAKZQ5u/5VdMZY3SqdUWw\ndkkFEGNVKfx3OAlO3ejZrcc+BS89Gw56p/3MRVbEoNWyIpikVYEXAeN6ttn+9eDjRXS7SdMXLfwB\nTlgapq1sH5TCP9pqwApA0iGUzqhJwLXA9sDlwKvqjRYxFq00vv/tExazPaI+rXQCH0mZCO5O27sC\nWwL/qDVVxBgk8XKYvGX/785f0N40Ea1VAAttLwCQNM72LcBL6o0VMXZITJD4CnAerH0yHNJnnv6D\nb4c5M/o9OKJGrfQB3FX1AZwLXCzpYeDOWlNFjBES0yiTt80CNrHf/6A05Vcw7YjS7DN/AcyZYc8+\nv9mk0Y0GNQpI0lTK6mAX2v5XC/ufBuwF3G9702rb8cDBwAPVbsfavrDPcRkFFKOaxJrAVyh9ZofZ\nXNxwpOgCdawHMLnnC/gLcB2wdouf/23gtX22GfiS7S2rrwv7OS5iVKrm6j8IuBH4G7BpCv/oVK00\nAZ0PzzyNOA7YgDI/ySYDHWj7Mknr9/NWru5jzJHYkLI04+rANJurG44UsUQD3gHYnmJ70+rrRZQR\nQVcM87xHSLpe0rckZYH5GNUklpH4AHAVcBGwXQr/GA1aehCsN9vXSHr5MM55MvDJ6vtPAV8E3t13\np6qvoMcs27OGcc6IWkhsRZm182Hg5Ta3NxwpukjVLzt1yMe3MBfQ+3u9XArYCljN9h4tnaA0Af20\npxO4lffSCRydTmIF4HjgIODDwOmZuC2aVsdsoBOBCdXXcsDPgNcPLR5IWqfXyzdSOssiRg2J3Sn/\nbicBm9n8Xwr/GI1qnQxO0pnALsAawFzgOMrtyhaUjuU7gMNsz+1zXO4AouNIrA58gTINyvtsft5w\npIhF1LEgzE8phXXPhy7yve29hxJ0gHOmAoiOISHgLcCXgbOA/7KZ12yqiOeqYzbQO4C1gO9RCv79\nKVfzM4eUMGIUkZhMGbgwGXijPewRcBEdo5U7gKttbz3QthENlTuAaJjE0sDhwMcpT/R+zmbAp98j\nmlTHHcAKkl5o+/bqBBtSVgiLGJMkNqUM7fwnsKPNnxqOFFGLViqAo4BLJd1RvV4fOLS2RBENkRgH\n/BdwGPBR4Js2TzebKqI+LY0CkjQOeCmlA/gW2/+sNVSagKLNJHamXPXPBo6wuafhSBGDVsdkcPsB\ny9m+DtgbOFPSVsPIGNExJFaROAU4AzjG5k0p/KNbtPIg2MdsPyppJ2A34DTg6/XGiqifxD6UK34D\nU+yMbIvu0koF8FT15+uAU23/DFi2vkgR9ZJ4vsSPgc8A+9u8x+aRpnNFtFsrncB/k/QN4NXAiVV/\nQCsVR0TjpCnTYNJ0mDgO5v8T9vozHP4Wytj+A2wWNp0xoimtPAewImVRlxts/7may2dT2xfVFiqd\nwDECSuG/w0lw6kbPbv3wQph3tP2/JzeXLKIeIz4VRJ8PP9T2N4aUbBBSAcRIkPa8EC7oZ9baaRfa\n5+/Z/kQR9apjNtDe3jvI/SMatPaa/W+fML69OSI602ArgFyVR8eTmCjxVVjrZf3vMX9BexNFdKbB\nVgCvqyVFxAiReB1wEzABbjgIDrlt0T0Ovh3mzGgiW0SnaaUTeFXgQMoUED2jhmx7em2h0gcQgySx\nFnASsA1wmM0lZfuUaTD5iNLsM38BzJlhzz6/0bARNaljPYDLgcspKyA9TWkGsu3ThxN0gHOmAoiW\nVHP1vxP4LOUhxU/aPN5oqIiG1DEb6PK2jx5GpohaSGwEnAKsDOxhc23DkSJGlVb6AM6QdKikdSSt\n1vNVe7KIxZBYRuJDwBXA+cD2KfwjBq+VO4CFwOcp0+P2TI1rYMO6QkUsjsTWwDeBB4DtbP7ScKSI\nUauVPoA7gG1t/709kdIHEM8lsSLwCeAdwAeB79q0/hRjRBeo40GwPwMZNx2NkXgNZRDC2pRZO7+T\nwj9i+FppAnocuE7SpZQl8qDmYaARABJrAF8Cdgbea3NBw5EixpRWKoBzq6/ecvUVtamGdh4AfBE4\nk3LVP7/ZVBFjz6Amg2uX9AF0L4n1KAsOrQscbHNVw5EiRo0Rfw6g12Lwvdl2RgHFiJFYGjiCsij7\nF4Ev2DzRbKqIsa2VJqBte30/DngzsHo9caIbSWxGGdr5GPAKm1sbjhTRFYbUBCTpGtu1LQyfJqDu\nIDEe+BhwMHAscFpG90QMXR1NQFvzbKfvUpTJtpYeWryIQmIq8A3gOmAzm/uaTRTRfVppAvoiz1YA\nTwJ3Avu18uGSTgP2Au63vWm1bTXgh8B6PZ9lOwtydwmJVYHPUZYZPdzmJw1HiuhatY4CkvRKYD7w\nnV4VwOeAv9v+nKQPA6vaPqbPcWkCGmOqoZ1vAr4KzASOtXm02VQRY8uIPQksaW9J6/d6fZykGyT9\nRNIGrXy47cuAh/ts3hvomUr6dOANrYaN0UliXUqh/0lgX5vDU/hHNG9JTUD/DbwcQNLrgLcDbwW2\npIzT7mex7ZasZXtu9f1cYK0hfk50mLL4yqTpMHEczFsIf5sBN0ymFPxfA95iP/M0eUQ0bEkVwNO2\nexbW2Af4lu2rgaslHT4SJ7dtSf22QUk6vtfLWbZnjcQ5ox6l8N/hJDh1o2e3fmgX+PlfYK9dbG5u\nLl3E2CRpKjB1qMcvqQKQpImUsdm7ASf3em/cUE8IzJW0tu37JK0D3N/fTraPH8Y5ou0mTV+08Af4\n3DiYdpe9Vwr/iBpUF8azel5LOm4wxy9pNtCvANcCVwN/tP376gRbAfcMNmgvPwEOqr4/iOfOMxSj\n0sTFXBRMGM7FQkTUaLF3ALZPk3QRsCZlrHaPe4F3tfLhks4EdgHWkHQX8HHgROAsSe9mEENKo3NJ\nrARrLGZgwPxMJR7RoRY7DFTShraXuNqSpBfavn3EQ2UY6KghsTfwNTjrZrjkRXBKrzmiDr4drphu\nzz6/sYARXWSwZeeSKoAfAitSmmz+QLnyF7AO5WngvYF5tt863ND9nDsVQIeTWJsypn8L4FCbWaUj\nePIRMGF8ufKfMyOFf0T7jFgFUH3YRpShnztSntwF+CvwG+DMge4QhioVQOeqHuj6d+AE4FTg03ZW\njIvoBCNaATQlFUBnkngRZf6eFYFDbK5vOFJE9FLHZHBv4rkrgP0DuNF2v0M4Y2yRWBb4APB+4NPA\nDJunmk0VEcPVymRw/w7sAFxK6QPYBbgG2EDSJ21/p8Z80TCJbSlz9d8DbGNzZ7OJImKktFIBLAts\n3DN9g6S1gO9Spon4NZAKYAySWBH4FGVt3qOBMzNXf8TYsqQHwXpM6jV3D5QndyfZfhD4Vz2xokkS\newCzKSu/TbE5I4V/xNjTyh3ApZJ+DpwFz0zpO0vSikDm8R9DJJ4HfBl4BXCYzUUNR4qIGg04CkjS\nUpTJ4HasNv0WOMc1Dh/KKKD2qoZ2vh34PPA94Dibx5pNFRGDNeKjgGw/Lek38Mw0vlfWWfhHe0ls\nQJnee01gL5urG44UEW0yYB+ApP2AK4F9q6+rJO1bd7Col8QyEkcDvwf+H7BdCv+I7tJKE9ANwO49\nY/4lPQ+4xPZmtYVKE1CtJDanDO18lNLWf1vDkSJiBIzYkpC9PxN4oNfrB6ttMcpIjJc4AbgI+F9g\n9xT+Ed2rlVFAFwK/kHQGpeB/C3BBralixEm8CjiFsr7DZjZzBzgkIsa4VpqARBkFtBNlSojLbM+s\nNVSagEaMxGqU0T2vBt5n87OGI0VETTIZXADPDO3cl7Ky29nAR23mNZsqIuo0YsNAJc3nuZPA9bDt\nlQYbLtpDYhLwP8CGwD42VzQcKSI60GI7gW1PsD1xMV8p/DuQxFISh1Mm67sK2CqFf0QsTiudwNGB\nyupbk6aXxdjnLYT1z4OT3wE8CbzS5pamM0ZEZ0sfwChUCv8dToJTN3p267FPwbonw38cafN0c+ki\noil1PAcQHWfS9EULf4ATlobzN0rhHxGtSgUwKq2yYv/bJ4xvb46IGM1SAYwyEm+A9bbp/935WZw9\nIlqWCmA01lKNAAALXElEQVSUkHi+xDnAibDUcXBInykcDr4d5sxoJFxEjErpBO5wEksBB1MWYz8F\n+G+bhaUjePIRpdln/gKYM8OefX6zaSOiSXkSeAyReAnwDWB54BCbGxuOFBEdLKOAxgCJ5SQ+CvyG\nMo3Djin8I2Kk5UGwDiPxcspc/XOArW3mNBwpIsaoxioASXdSFiR5CnjC9nZNZekEEhMo7fxvAY4C\nfmgvdi6miIhha/IOwMBU2w81mKEjSEyjLNByKTDF5sGGI0VEF2i6CairO3ol1qRM1/xy4GCbXzYc\nKSK6SJOdwAZ+KekPkg5pMEfbSUjiIOBG4C5g0xT+EdFuTd4B7Gj73mqR+Ysl3WL7sgbztIXEC4Gv\nA6sBr7W5tuFIEdGlGqsAbN9b/fmApJnAdsAzFYCk43vtPsv2rLYGHGESy1A6dz8MnACcZPNks6ki\nYjSTNBWYOuTjm3gQTNIKwNK250laEbgI+ITti6r3x9SDYBJbUYZ2/h14j81fGo4UEWPQiC0JWbO1\ngJllvXmWAb7fU/iPJRIrAJ8ADgQ+CHw3QzsjolM0UgHYvgPYoolzt4vE7pS5ey6nDO18oOFIERGL\naHoY6JgjsTrwRUq73HttLmg2UURE/zIX0AiphnYeAMwGHqZc9afwj4iOlTuAESCxHnAy8ALg9TZX\nNRwpImJAuQMYBomlJY4ErqbM3Ll1Cv+IGC1yBzBEEpsBpwILgFfY3NpwpIiIQUkF0IKy+tak6TBx\nHDz2L3jLQ3Dgq4CPAKfZPN10xoiIwcqKYAMohf8OJ8GpGz279QPz4c732md/r7lkERGLyopgI27y\nkYsW/gBfmACPv62ZPBERIyNNQP2o5u2ZCuwLW+/a/14TxrcxUkTEiMsdQEViWYk9JE4F7gU+A9wG\nN13R/xHzF7QxXkTEiOvqOwCJ5YDdgTcDrwdupSzCvo3NX8s+t94Eh6yzaDPQwbfDnBntTxwRMXK6\nrhNYYnngNZRC/9+APwI/As6xuav/Y6ZMg8lHlGaf+Qtgzgx79vl15IuIGKrBlp1dUQFIjAP2APYF\n9qKsxHU2pdD/20idJyKiSakAnvkMxgN7Uq70pwHXUgr9H9vcO/yUERGdpasrAIkVKYX9m4HXAn+g\nNO/MtJk7sikjIjpL11UAEhMozTr7Aq8GrqRc6c/MHPwR0U26ogKQWAl4HaXQ3w34LaXQP8/m7+1J\nGRHRWcbMk8DSnheW0Tc9r1lZ4h0S5wF3A28DzgPWt9nT5lsp/CMiWtexdwBg4LC/wMbnwn++BNgZ\nmEW50v+JzSNNZoyI6DRjpgmIZ9ZO/8/74StHAz+z+UeTuSIiOtlgK4BR8CTwPX+0+X7TKSIixpqO\n7QN4VubciYioQ4dXAJlzJyKiLh3cBDTtwsy5ExFRn47tBO6UFcEiIkaLMfMcQERE1CsVQEREl0oF\nEBHRpRqpACS9VtItkv4s6cNNZIiI6HZtrwAkLQ18jTJd88uA/SVt3O4cQyFpatMZ+kqm1iRT6zox\nVzLVo4k7gO2A22zfafsJ4AeU9XhHg6lNB+jH1KYD9GNq0wH6MbXpAP2Y2nSAxZjadIB+TG06QD+m\nNh1guJqoANaFRdbevbvaFhERbdREBdB5Dx5ERHShtj8IJml74Hjbr61eHws8bfuzvfZJJRERMQQd\nPR20pGWAP1FW8roHuArY3/Yf2xokIqLLtX0uINtPSvoP4BfA0sC3UvhHRLRfR84FFBER9euoJ4El\nnSZprqQbm87SQ9IkSZdKuknSbEnTOyDTOElXSrpO0s2STmg6Uw9JS0u6VtJPm87SQ9Kdkm6ocl3V\ndB4ASatIOlvSH6u/w+0bzvOS6vfT8/WPDvm3fmz1f+9GSWdIWr7pTACSjqwyzZZ0ZEMZnlNeSlpN\n0sWSbpV0kaRVlvQZHVUBAN+mPCDWSZ4AjrK9CbA9cHjTD67ZXgjsansLYDNgV0k7NZmplyOBm+ms\n0V4Gptre0vZ2TYepnAScb3tjyt9ho82gtv9U/X62BLYGHgdmNplJ0vrAIcBWtjelNBm/tclMAJKm\nAAcD2wKbA6+T9MIGovRXXh4DXGz7xcAl1evF6qgKwPZlwMNN5+jN9n22r6u+n0/5j/r8ZlOB7cer\nb5ej/Md4qME4AEh6ATAN+CbQadN5d0weSSsDr7R9GpR+MdudtN717sDttu8acM96PUq5AFuhGjyy\nAvC3ZiMB8FLgStsLbT8F/ArYp90hFlNe7g2cXn1/OvCGJX1GR1UAna66ItkSuLLZJCBpKUnXAXOB\nS23f3HQm4MvAB4Gnmw7Sh4FfSvqDpEOaDgNsADwg6duSrpF0qqQVmg7Vy1uBM5oOYfsh4IvAHMqI\nwUds/7LZVADMBl5ZNbesAOwFvKDhTD3Wsj23+n4usNaSdk4F0CJJE4CzgSOrO4FG2X66agJ6AbBz\n0/OSSHodcL/ta+mgq+3KjlXTxp6UJrxXNpxnGWAr4H9tbwU8xgC36u0iaTng34AfdUCWFwL/CaxP\nueueIOltjYYCbN8CfBa4CLgAuJbOu+jBZYTPEptiUwG0QNKywDnA92yf23Se3qqmg58D2zQc5RXA\n3pLuAM4EXiXpOw1nAsD2vdWfD1DatZvuB7gbuNv276vXZ1MqhE6wJ3B19btq2jbA72w/aPtJ4MeU\nf2eNs32a7W1s7wI8Qnm2qRPMlbQ2gKR1gPuXtHMqgAFIEvAt4GbbX2k6D4CkNXp69yWNB15NuQpp\njO2P2J5kewNKE8L/s31gk5kAJK0gaWL1/YrAa4BGR5nZvg+4S9KLq027Azc1GKm3/SkVeCe4Bdhe\n0vjq/+HulAEGjZO0ZvXnZOCNdECTWeUnwEHV9wcBS7xg7ahF4SWdCewCrC7pLuDjtr/dcKwdgbcD\nN0jqKWSPtX1hg5nWAU6XtBSlEv+u7UsazNOfThkFtBYws5QfLAN83/ZFzUYC4Ajg+1WTy+3AuxrO\n01NB7k4ZedM429dXd5F/oDSxXAN8o9lUzzhb0uqUTur32X603QF6lZdr9JSXwInAWZLeDdwJ7LfE\nz8iDYBER3SlNQBERXSoVQEREl0oFEBHRpVIBRER0qVQAERFdKhVARESXSgUQY5Kk+dWf60naf4Q/\n+yN9Xv92JD8/ol1SAcRY1fOAywbAAYM5sJp5ckmOXeRE9o6D+fyITpEKIMa6EykzN15bLeKxlKTP\nS7pK0vWSDgWQNFXSZZLOo8z2iKRzqxlEZ/fMIirpRGB89Xnfrbb13G2o+uwbqwVo9uv12bMk/aha\nAOZ7PeEknVgteHK9pM+39TcTXa+jpoKIqMGHgQ/Y/jeAqsB/xPZ21epSv5HUMzXElsAmtv9avX6X\n7Yer+ZauknS27WMkHV7NLtqj525jH8oCIZsBzwN+L+nX1XtbAC8D7gV+K2lHylw3b7D90irbSjX8\n/BGLlTuAGOv6Tk39GuDAal6nK4DVgI2q967qVfgDHFmtuXA5MAl40QDn2gk4w8X9lIVCtqVUEFfZ\nvqeaovc6YD3KLJILJX1L0huBBUP+KSOGIBVAdKP/6Fn+0PYLey0y8ljPDtX6CrsB21frLlwLjBvg\nc81zK5yeu4N/9tr2FLBstZrUdpTpoF8HNDnBYHShVAAx1s0DJvZ6/QvgfT0dvZJevJjVuFYCHra9\nUNJLKetB93hiMR3FlwFvqfoZngfsDFzFYhbIqWbfXMX2BcDRlOajiLZJH0CMVT1X3tcDT1VNOd8G\nvkpZYeqaao75+ynzufddPelC4D2SbqYs9nF5r/e+QZke/Grb7+g5zvZMSTtU5zTwQdv3S9qY506P\nbUrFdJ6kcZRK4qgR+ckjWpTpoCMiulSagCIiulQqgIiILpUKICKiS6UCiIjoUqkAIiK6VCqAiIgu\nlQogIqJLpQKIiOhS/x84XvgIqzC1dAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10895d5f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEZCAYAAABxbJkKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm85nPdx/HX2zqYQaIQskdNlkKLZEolE9qVSqkbdefG\nnVJo4W6xVdyGuhMJJWWPjK0ykhQxZFAiy1inss1Yh3nff3x/x1xznOU6Z851/a5zrvfz8TiPc12/\n7fqcM3N+n993l20iIqI7LVJ3ABERUZ8kgYiILpYkEBHRxZIEIiK6WJJAREQXSxKIiOhiSQJdTNJJ\nkr7R5LHTJP1H9fqjki5ubXSdS9JWkv5adxwRIyFJYAyqbtgPSVpikENdfTXj+WNtn2p724WJcbgk\n3Slpmzo+u4ftK2xvUGcMPSRNkjRzIa+xjaS/Snpc0m8lrTHAsStIOkfSnOrfYuehXEvS4ZL+VX0d\n1mvfNyTdKGmupIMW5meK5iUJjDGS1gS2AGYBOzZzSivjaYGhJK5hkdQRfxeqtPgzVgTOAr4MvAj4\nM/CLAU75HvAU8BLgo8D/SXplM9eS9Gng3cBG1dcO1bYefwf2Ay6gxf/GMV9H/GePEfVx4NfAT4BP\nNO6QtKmk6yQ9JunnwLiGfS+S9CtJs6pSxPmSXtbXB0jaVdIVDe/nSfq0pFslPSzp2F7Hf0rSzdV1\nLxroSXO4qvvl/pJuq54yfyHpRQ37z5B0v6RHJF3ec+Oq9p0k6f8kTZU0B3hL9ZT7eUk3VOf8XNKS\n1fELPH0PdGy1/4uS7pN0j6Tdqt/X2v38HNMkfVPSlcDjwNqSPln9/h6TdLukPapjlwEuBFaVNLva\nv/Jgv4te3gfMsH2W7WeAg4GNJa3fR2zLVMd/1fYTtq8Efgns0uS1PgF8x/Z9tu8DvgPs2nN926fY\nvgiYzeh7OBm1kgTGno9Tnr5OB7aV9BIAlaqhc4GTKU9pZwDvZ/4Tl4AfAWtUX08CC9zMB/EuYDPK\nE95OkratPvfdwAHAe4EVgSuA04b/4/Vrb0rJ583AKsDDlKfWHhcA6wIrAdcBp/Y6f2fgG7bHA7+n\n/F4+CGwLrFX9XLv289n9HivpncDngG2A9YBJDP6U+zFgN2A8cBfwIPAu28sCnwSOkrSp7ceBdwL3\n2Z5ge1nbDzTxu2j0KuCG538Q+wngNmBiH8euDzxr+7aGbTdU1xjoWj37X9m4H/hLw76oSZLAGCLp\nTcDLgPNs/x24GfhItfv1wGK2j7b9nO2zgGt6zrX9kO1zbD9lew5wCLD1ED7+MNuP2Z4JXAZsXG3/\nDHCo7b/ZngccCmwiafWF+Vn78GngK9VT5lzgf4AP9FTt2D7J9uMN+zaWNKHh/HNtX1Ud+3S1bYrt\nB2w/DJwPbDLA5/d37E7AibZvsf0kcBADP+UaOKk6fp7tZ21PtX1HFdvvgEuArarj+7rWgL+LXpYB\nHuu17TFKAuptfB/HzgYmDLD/sV77H23ic6KNkgTGlk8Al9ieXb0/g/lVQqsC9/Y6/i6qm4ikpSUd\nV1VtPApcDiw3hDrpBxpeP8H8P+6XA0dX1UQPA/+utr+gqknSD6pqjdmS9m/yc3usCZzT8Dk3A88C\nL5W0qKTDquqRR4E7qnNWrL4b6KtxtfFnepKBb1i9j12mer1Kr2vf08TPskAskraT9EdJ/65+tsnA\niwc4f036+V30cewcYNle25aj3NybPbbnxj97kGv1Pn+5alvUaLG6A4iRIWkpylPnIpLurzYvSbmR\nbwTcxwtvvC+nFNcBPk8p7m9he5akTSjVJmLhGunuplSzDFoFZPszlJLDcD/nkz1P840k7UKpHtnG\n9l2Slgceoj31zvcDjaWeZkpAz/++q7aFsyhVRL+0/Zykc5gfe1//Nv3+LvpwEw1tR1W9/zrV9t5u\nBRaTtG5DldDGDccOdq2bKCWkPzecO6OfuNIw3CYpCYwd76E87W1I+ePauHr9e0o7wVXAs5L2lrS4\npPcBmzecP57yBPuopBUo1RbDJebfpH4AHKj5PUiWk/TBhbg2wBKSxjV8LVZ9ziE9jc6SVpLU0ztq\nPPA08FB1Yzqkj3hHWs81Twc+KWkDSUsDXx3CuQBLVF//AuZJ2g54R8P+B4EXS2p8wh7od9HbOcBE\nSe+TNI7y73697Vt7H1i1QZwNfL0qOb4J2IHSCaGZa50C7CtpVZVOB/sCJz3/Q0uLVectCixe/dvm\nHtVi+QWPHR+n1D3fY3tW9fUgpXH3I8BzlN4bu1KqZHaiPGH2+F9gKcrN5g+UXif9PY317qbZ+7jG\nMQXnAocDP6+qYm6kNKAujKmUKqeer68BRwPnAZdIeoyS9Laojj+FUvV1L+XJ86o+4h/syXOwn7nP\nY6veLlMo7SS3Vp8NJSkNdD7V+bMpDb2nU0ovO1N65PTs/yulof0fKr2vVmbg38WCH2T/i9JB4FvV\n9TcDPtyzX9KBkqY2nPJZyv+TWcBPgc/YvqWZa9k+jtJeciOlUfh82z9suPYJlH/PD1O6mT5BKQFF\nC6nVi8pIupNSZ/gcMNf2FtWT5i8o1RF3AjvZfqSlgUR0AEkbUm6CS1QN5RG1akdJwMAk25va7nka\n2R+41Pb6wG+q9xFjkqT3Slqy6qt/OKX3VhJAdIR2VQf1rnPdkdJfner7e9oUR0Qd9qDU3d8GzAX+\ns95wIuZrR3XQPyh9g58DjrN9vKSHbb+o2i/goZ73ERHRPu3oIrql7fslrQRcql6zL9q2pHQHi4io\nQcuTgO37q+//rPo3bwE8KGll2w9IWoXS02ABSQwREcNju/luz7Zb9gUsDUyoXi8DXEnp43wE8KVq\n+/6UKQd6n2tw9bXdha2Mcwg/z8F1x5CYxlZciSkxtSAuD+X4VpcEXkoZvg6l1HGq7Usk/Rk4XWWR\nkjspfdb7sf9z8JIzWxxnRERXamkScJn06gWTbtl+CHjb4FeYfBG86x9w2H4S59g8NPJRRkR0r46e\nO8ieuh2AxHeAcyTeYQ840rLVptX42f2ZVncAfZhWdwD9mFZ3AH2YVncAfZhWdwB9mFZ3AH2YVncA\nI6HlXUSHS5JdNW5ILEKZEfMJ4ON2JpeKiOhL472zGaNi7iCbeZTVi9anrFYUEREjYFQkAQCbJygj\njXeRFlw2MSIihqej2wR6s3lQ4l3ANIm7bS6rO6aIiNFs1JQEetjcQplq9ucSG9YdT0TEaDbqkgBA\nVQL4InCB1OeSeRER0YRRmQQAbE6mrGh0nsTSdccTETEajYouov0fgyirRi0NfLDqRRQR0bXGZBfR\n/lTjBXYDVqTMRxQREUMwqpMAQDWC+L3A9lIW64iIGIpR1UW0PzYPVV1Hfy9xl83UQU+KiIjRXxLo\nYXM78D7gJOmFk9ZFRMQLjZkkAGBzFbAncL7EanXHExHR6cZEdVAjmzMk1gJ+JbGVzey6Y4qI6FSj\nuoto/+ci4AfAGsAONs+OaHARER2qq7qI9qfqOronIOCYKilEREQvYzIJAFRP/zsBWwKfrzmciIiO\nNObaBBrZPFZ1Hf2DxB02Z9UdU0REJxnTSQDAZqbEjsAlEvfa/LHumCIiOsWYrQ5qZDMd2JWyTvHa\nNYcTEdExuiIJANhcAHyTMv30i+qOJyKiE4zJLqIDX5cjgU2BbW2eGenrR0TUaaj3zm5MAosCZwBz\ngE9U3UkjIsaEjBMYhM1zwMeADYCv1RxOREStxnzvoL7YPFH1GPqjxD9sflJ3TBERdejKJABg80A1\nhuAyibttLq87poiIduu66qBGNjcBOwOnS7yi7ngiItqtq5MAgM1vgP2BqRIr1R1PREQ7dX0SALD5\nMXAacJ7EUnXHExHRLl3XRbT/z0PAqcDiwIds5rXrsyMiRkq6iA5TNV7gk8DKwKE1hxMR0RZJAg1s\nngbeA7xXYo+644mIaLWu7SLaH5t/S0wGfl91Hb2o7pgiIlolJYE+2NwGvB84RWKjuuOJiGiVJIF+\n2FwJ7EVZsH7VuuOJiGiFllcHSVoU+DNwj+0dJG0BHEvphfMs8Fnb17Q6juGw+UVZf+Ci30nH3gFL\nLw6zn4KZU+wZU+uOLyJiYbWjTWAf4GZgQvX+COCrti+WtF31/i1tiGOYJt4A264Ev1pn/rbd15Em\nkkQQEaNdS6uDJK0GTAZOAHr6rd4PLFe9Xh64t5UxLLzV94bvLrvgtuPXhTX2qieeiIiR0+qSwFHA\nfkDjTXR/4PeSvkNJQm9ocQwLacK4vrePz8jiiBj1WpYEJG0PzLI9XdKkhl0/Ava2fY6kDwInAm/v\n5xoHN7ydZntai8IdwOyn+t6++NLtjSMi4oWq++ukYZ/fqmkjJB0C7EJp/B1HKQ2cDbzb9rLVMQIe\nsb1cH+e3ddqI/kgTJ8Mbji5VQD0+ex+8fxxscwRwRFYni4hO0ZHLS0raGvhC1TvoOuBzti+XtA1w\nmO3N+zinI5IA9CSCNfYqVUBznoS7j4EZNwJnAjOBT9rMrjnMiIgh3zvbOWK4J9vsAXxP0pLAk9X7\njlb1AnpBTyCJNwNTgKsl3mvz17YHFxGxEDKL6AiQ2I0y6dweNufUHU9EdK+OrA4ajtGUBAAkNqdU\nD/0M+Eq1oH1ERFslCdSoWpns58BzwEds/lVzSBHRZbKeQI1s/glsC1wPXCPxmppDiogYUJLACLN5\n1uaLwBeBiyV2rTmkiIh+pTqohSReCZwD/Ab4b5tnag4pIsa4VAd1EJubgS2AVYHLJV5Wc0gREQtI\nEmgxm0eB9wHnU9oJ3lxzSBERz0t1UBtJbAucQhlTcHSmm4iIkZYuoh1OYi3KHEq3ALvbPF5zSBEx\nhqRNoMPZ3AG8EZgLXCWx7iCnRES0TJJADWyeBHYFfgD8QeJd9UYUEd0q1UE1k3gjcDpwPPANm3k1\nhxQRo1jaBEYhiZWBM4BHgV1sHq45pIgYpdImMArZPAC8Fbid0o10o5pDiogukSTQIWzm2uwDHAT8\nRuIjdccUEWNfqoM6kMTGlG6k5wFftJlbc0gRMUqkOmgMsLkB2Ax4BfDrqs0gImLEDZgEVKzermBi\nvqpxeHtgGqWd4A31RhQRY1EzJYELWx5F9Mlmns1BwGeBX0r8p0RXVpFFRGsMmARcGgyulbRFm+KJ\nPticD2wJ7AmcKLFUzSFFxBgxaMOwpL8B6wJ3wfPz3Nh2S7sxdnPDcH8kxgMnAOvBx46Bf38YJoyD\n2U/BzCn2jKl1xxgR9RrqvXOxJo7Ztvreky1yY66JzRyJneGHP4TVT4CfLjp/7+7rSBNJIoiIoRi0\nTcD2ncDywI7ADsBy1baoQZl++pzV4dBFF9xz/Lqwxl71RBURo9WgSUDSPsBPgZWAlwI/lbR3qwOL\ngUwY1/f28WkriIghaaY6aDfgdbYfB5B0GPBHYEorA4uBzH6q7+0rrCaxpM3T7Y0nIkarZgeLzevn\nddRi5hTY/bYFt336TnjnLOD6ambSiIhBNVMS+DHwJ0lnUxqF3wOc2NKoYkD2jKnSRGDyXqUKaM6T\ncPcxcNyFwPuBMyXOBA60mVNvtBHRyQbsIippEeANwFPAmyg9hK6wPb3lgaWL6LBJrAAcCUwCPm1z\ncb0RRUS7jPh6ApKut73JQkc2REkCC69a2P44ytQT+9o8VG9EEdFqrZhA7teSPiApN+RRpioBTKQs\nVjND4gOZdiIiGjVTEpgDLA08R6kWgjJieNmWBpaSwIiS2JIy2vgWYE+b+2sOKSJaYERLAlWbwLa2\nF7G9uO0J1VdLE0CMPJsrgU2Bm4EbJD6VUkFEpE2gC0lsAvwIeBjYw+YfNYcUESMkbQIxKJvrgdcB\nFwNXS/y3xKKDnBYRY1DaBLqcxHqUtoIlgf+wuanmkCJiIYx4ScD2+IVpE5C0qKTpks5v2LaXpFsk\nzZB0eLPXipFn83fgLZRBgdMkviaxRM1hRUSbNDOB3CKSdpH0ter9GkNcZGYfSmOkq/PfQpmRdCPb\nE4HvDD3sGEnVCmbHURqOtwCulchCQhFdoJk2ge9TRg1/pHo/p9o2KEmrAZMp1Q09xZP/BA61PRfA\n9j+HEnC0js09lOnCDwHOk/iuxNI1hxURLdRMEnid7c8CTwLYfghYvMnrHwXsx4KTzq0HvFnSHyVN\nk7TZUAKO1rKxzWnAq4GVgRsl3lpzWBHRIs0kgWckPd9zRNJKNDGTqKTtgVnVPEONjRSLAS+y/XpK\ngjh9aCFHO9j80+ajlOq8kyWOl1i+7rgiYmQ1M4voMcA5wEskHQJ8APhKE+e9EdhR0mRgHLCspJ8A\n9wBnA9i+RtI8SS+2/e/eF5B0cMPbabanNfG5MYJsfiXxO+BwytQTe9r8su64IqKQNIkyWeTwzh+s\ni2j1IRsC21Rvf2P7liF9iLQ18AXbO0j6NLCq7YMkrQ/82vYafZyTLqIdRmJr4HhgOrC3zYM1hxQR\nvbRisBi2b7F9LDB3qAmg8TLV9xOBtSXdCJwGfHyY14s2s7kc2Bi4A/iLxC6ZeiJidGuqJPD8wdJ0\n25u2MJ7Gz0pJoINJvJYy9cT9wGds7qo5pIhg6PfOZtoEFrj+EI+PMcrmWonNKY3710ocBBvdCS/b\nCyaMK+sgz5xiz5hac6gRMYChlgRWs31PC+Np/KyUBEYJiQ3ggrPg8rXhiHHz9+x+G1y1TxJBRPuM\n2Mpikj7f8NbMLwUYwPaRww2yqcCSBEYVafLFMPUdL9wz+SJ76nbtjyiiO41kddAEyg3/FcDmwHmU\nRLA9cPXCBBlj0fgl+9m+VHvjiIih6DcJ2D4YQNIVwGtsz67eHwSkeB+9zH6q7+2rbiixdtYsiOhM\nzXQRfQkwt+H93GpbRIOZU0obQKPd/wGvvhC4RuLrmYcoovM00zvoFOBqSWdTqoPeA5zc0qhi1LFn\nTJUmApP3KlVAc56Eu4+xj58qsTrwbeAWic8DZ9k03yMhIlqm2RHDrwXeVL39XTUfUEulYXjskXgL\nMAV4kDLi+OaaQ4oYc1oyYpiysths20cD90haa1jRRVezuYyyZsF5wOUSR0osV3NYEV2tmUVlDga+\nCOxfbVoC+GkLY4oxzOZZmynAq4Blgb9KfEJq+oEkIkZQM3947wXeDTwOYPteSvfRiGGzmWWzG+X/\n1p7AldVUFBHRRs0kgadtP79+gKRlWhhPdBmbq4HXU2YnvUDihxIr1hxWRNdoJgmcIek4YHlJewC/\noSwXGTEiqjWOTwQ2oKxgd7PEntKQ57aKiCEasHeQJAGrU/44e6YEuNj2pS0PLL2DupbEqym9iFYA\n9rL5Xc0hRYwaIzZ3UHUxATfanjgSwQ1FkkB3q9Yp+CDwHeD3wH4299YbVUTnG9Euoi4Z4lpJWyx0\nZBFDUC14fzqwIWURmxskviTRzxxFETEcgw4Wk/Q3YF3gLqoeQpT8sFFLA0tJIBpIrAscRZnQcB+b\nC2sOKaIjjWh1UHXBNfvabvvOoQQ2VEkC0ReJdwH/C9wCfM7m9ppDiugoIz5i2Pad1Q3/CWBew1dE\n29lcAEwErgL+JPGNTEwXMXzNjBjeUdLfKfWylwN3QoriUR+bp20OBTahVFXeIvHBLHofMXTNjBP4\nJvAG4FbbawHbAH9qaVQRTbC5x2Zn4BPA14BfS7yq5rAiRpVmksBc2/8CFpG0qO3LgM1aHFdE02ym\nUSamOxeYJnFUJqaLaE4zSeBhSROAK4BTJU0B5rQ2rIihqSamOwZ4JTCeMjHdJyUWkSZOlra7SNpp\nWvk+cXLN4UZ0jGZ6B42nDOVfBPgoZebHU23/u6WBpXdQLASJzYFjYeqy8Kvx8P3V5u/d/Ta4ah97\nRpZJjTFnxLuI1iVJIBZWmZ561xvgpD5GvE++yJ66Xfujimitod47B52gS9IceH4pwCWAxYE5tpcd\nXogR7WEzT3qinxLr+KXaG01EZxo0Cdge3/Na0iLAjpSpfyNGgdlP9b19qeUllLWOo9sNaTUn2/Ns\nnwu8s0XxRIywmVNKG0Cjz94LO48H/iDlgSa6WzPVQe9veLsI8FpKQ3FEx7NnTJUmApP3KlVAc56E\nu4+B718E7AKcKXEFcIDNnbUGG1GDZnoHncT8NoFnKSOGj7c9q6WBpWE42kBiGeALwN6U1c0OsXms\n3qgihi+9gyKGQeJllNHx2wEHAT+yebbeqCKGrhWziB5DKQn0XHSB17b3Hk6ggwaWJBA1kHgN8F1g\nJeDzNhfXHFLEkIz4LKLAOOA1wK3A3ynD85cA/gxcO5wgIzqVzXXAW4EDgWMkLsx8RDGWNVMS+BPw\nJttzq/eLA7+3/bqWBpaSQNRMYgngM8BXgLOAg2xa2hYWsbBaURJYnjJVRI8J1baIMc3mGZspwAaU\nHnE3S+wvMa7m0CJGTDNJ4DDgOkknSToZuA44tLVhRXQOm4ds9qVMqb4FZXK6D2f9ghgLmuodJGkV\n4HWURuE/2X6g6Q+QFqW0H9xje4eG7Z8Hvg2saPuhPs5LdVB0JIlJwJHA08C+NlfVG1HEfCNeHSRp\nS2B2NVJ4WeCLkl4+hJj2AW5m/lgDJK0OvJ2yeH3EqFKtX7AZ8APgDImfS6xZa1ARw9RMddAPgCck\nbQzsC9wOnNLMxSWtBkwGToAFis5HAl8cWqgRncNmns3JwCsoDznXShwmkYkVY1RpJgk8a3se8B7g\ne7a/R2kcbsZRwH40LEwv6d2UqqG/DDXYiE5j87jN14FXAy8B/ibxaWnwKVkiOkEzSWC2pAOBjwG/\nqur4Fx/sJEnbA7NsT6cqBUhamtL/+qDGQ4ccdUSHsbnP5lOUku+HgRukTLQYna+ZcQKrADsD19i+\nQtIawCTbA1YJSTqEMkHXs5QBZ8sCFwJbAU9Uh60G3Ats0XsuIkkG/qdh0zTb05r8uSJqU/Ua2gH4\nDvAP4As2M+qNKsYqSZOASQ2bDmrZ3EGStrf9q6ZPmH/e1sAXGnsHVdvvAF6b3kExFvUabHY2ZbDZ\ng/VGFWNdKwaLNfrGEI9v1Fe26czZ6yJGQK/BZk8AN/UMNpMmTi6L3u80rXyfOLnmcKNLDbUkMN32\npi2Mp/GzUhKIMUViPeBw+PWW8Mt5cMzK8/fufhtctY89Y2ptAcaY0NKppCVtYfvqYUU2REkCMVZJ\nO18Np23+wj2TL7Knbtf+iGIsacVC84sB7wLWBBaTtBVlCukjhx1lRFd77om+t49fqr1xRDSRBIDz\nKZNn3UhDf/+IGK7ZT/W9/SVrS6xs0/S0LBELq5kk8DLbG7U8koiuMXMK7L4OHL/u/G173AFbX09p\nPD4G+K7N7LoijO7RzDiB7wCX2m7rCktpE4ixrPQGWmOvUgU050m4+xh7xtRqDqJvAtsAXwdOsJlb\na7AxqrRiecn3AT+ldCft+c9o2y2dIyVJILpZtczl4cAawAHAOXa6VMfgWpEE7gR2BGZUcwi1RZJA\nBEi8AziCMs7giza/rzmk6HCtGCx2N3BTOxNARBQ2l1DW+P4/4FSJcyU2rDmsGEOaKQmcDKxFmffn\nmWpzy7uIpiQQsaBqWcv/Ar5EmYbiYJv7640qOk0rSgJ3AL8FlgDGU6aRbnYq6YgYITZP2XyHsobB\nY8AMia9L+XuM4RvSiOF2SkkgYmASL6fM5/UOSo+iH9rPl9ajS7WiYfiyPjbb9luHGtxQJAlENEdi\nE0pPorUp63WcmZ5E3asVSWCzhrfjgPdTVhvbb3ghNhlYkkDEkEi8ndKT6GlKT6Lf1RxS1KClE8g1\nfMg1tvuYAGvkJAlEDJ3EIpRFoL4F/AXY3+bmeqOKdhrxhmFJKzR8rSjpnZDFtCM6kc08m1MpaxhM\nA6ZJHC+xar2RRadqdrBYz0HPAncC/2O7pYNWUhKIWHgSLwL2B3YDfgAcYfNovVFFK41YSUDSFpJW\nsb2m7bUo6/3+tfpK8TJiFLB52OZLwKbAy4BbJfaulr6MGLA66DhKAxOS3gwcCpwEPAr8sOWRRcSI\nsbnbZlfg7cB2wC0SH5JIabvL9VsdJOkG2xtXr78H/NP2wb33tSywVAdFtIzENpSeRM9RehJNKzOb\nrr43TBhX1jyYOSXLXY4+I7my2KKSFrc9F3gbsEeT50VEh7P5jcTmwIeAH0u//Cds9VL4vzXmH7X7\nOtJEkgjGtoGqg04DLpd0HmUGwysAJK0HPNKG2CKihaqeRKcBG8D5ExZMAFAWvVljr1qCi7bp94ne\n9rck/RZYGbikYRZRAfmPETFG2DwtPfYgpVtpL1n3eKwbsFrH9lV9bLu1deFERD36W/f4RatKLGPz\neHvjiXZpZhbRiBjzZk6B3W9bcNtn7oZt7wduk9gz3UrHpswiGhHAgOsebwocQpnC+qvAaTZZZKpD\ntWXuoHZIEojoLBKTKOOFlqbMVjo1s5V2niSBiGiZanDZjpSSwUPAAVn3uLO0YmWxiAgAbGzzS2Aj\n4ATKuse/ktio5tBimJIEImLIbJ6zORlYH7gUuETipxJr1xxaDFGSQEQMm83TNkcD6wG3AtdIHCux\ncs2hRZOSBCJiodnMtvk6ZcDZM8BNEt+UWK7m0GIQSQIRMWJs/mmzL/AaYFXg7xJfkMjI4w6VJBAR\nI87mLptPAZOAN1LWMdhNyuSTnSZdRCOi5SReBxxGKR18BTgrA85aI+MEIqIjVWMM3k4ZcGbgAODX\nGXA2spIEIqKjSSwCfAD4JjCTMuDs6nqjGjs6crCYpEUlTZd0fvX+25JukXSDpLMlpQdBRJeo1jE4\nHXgV8HPgbImzJDasObSu1K6G4X0oi9P3FDsuAV5VLVF5K6VYGBFdxGauzfGUMQZ/BC6X+JHEGoOc\nGiOo5UlA0mrAZMoQcwHYvrRhkZo/Aau1Oo6I6Ew2T9p8mzL6+AFgusSREitKEydL210k7TStfJ84\nueZwx5x2dNc6CtgPWLaf/Z+iLGUZEV3M5hHgyxLHAl+B3/4D3j4Xjlph/lFZ93iktTQJSNoemGV7\nuqRJfez/MvCM7Z/1c/7BDW+n2Z7WijgjonPY3A/sKR37ajh7qwX3Hr8uTN4LSBKoVPfWScM9v9Ul\ngTcCO0qaDIwDlpV0iu2PS9qVUk20TX8n2z64xfFFRMdarJ9xBOOXbm8cna16OJ7W817SQUM5v6Vt\nArYPtL29mLnuAAAK6UlEQVS67bWADwO/rRLAOylVRO+23c/aphHR3fpb93itzSQ+KrFoe+MZm9o5\nbYSY3zvoGGA8cGnVdfT7bYwjIkaFvtY93u12WOKbwH8BN0i8rxqEFsOUwWIR0bEGWPdYlOrkbwLP\nUaaiuDijjzNiOCK6SDX6+P3A14F/AV+2+V29UdUrSSAiuk7VPvBR4GDgNuAr3ToVRUdOGxER0UrV\ncpenUBa1OYsyFcW5Wft4cEkCETFm2DxjcxxlKorLKWsfnyaxfs2hdawkgYgYc6qpKI4C1gVmAFdW\n8xK9vObQOk6SQESMWTZzbL5FmZfofuA6iWMlVqk5tI6RJBARY57NwzZfATYEngFukjhC4sU1h1a7\nJIGI6Bo2s2z2BV4NTKCsffw/El27pkmSQER0HZt7bf4T2Bx4OfB3iS9JLFNzaG2XJBARXcvmHza7\nAlsDrwVuk9hbYsl6I2ufJIGI6Ho2t9jsRJmK4h2UksFuEovXHFrLJQlERFRspttsD3wI2Bm4eazP\nWJppIyIi+iHxVuBblEbkrwLnwsTtYPW9YcK4Mt31zCmdtNJZ5g6KiBhBC85YeuEycMF4OLZhnMHu\nt8FV+3RKIkgSiIhogTJj6cevg1M2fuHeyRfZU7drf1QvlAnkIiJawGYePPVI33vHL9XeaEZOkkBE\nRNP6W/JytVdJvK69sYyMJIGIiKb1teTl7rfDOr8AzpC4QGKzWkIbprQJREQMwQBLXi4J7AYcCPwZ\nONhmevvjS8NwRERtJJYC9gC+BFxFSQY3tu/z0zAcEVGbai2DoylrGVwJXCpxusSrag6tT0kCEREt\nYPOEzZHAOpTqocskfiaxQc2hLSBJICKihWwetzmCkgxuBK6Q+InEejWHBiQJRES0hc1sm0MpyeBW\n4A8SP5ZYu864kgQiItrI5jGbbwDrAXcBV0ucILFmHfEkCURE1MDmEZuDmb/+8bUSP5BYo51xJAlE\nRNTI5iGbrwKvAB4Gpkt8T+Jl7fj8JIGIiA5g8y+bA4ANgSeAGyWOllhlkFMXSpJAREQHsZllsx/w\nSuA54CaJIyVe2orPSxKIiOhANg/Y7AtMBBYDbpE4QmKlkfycJIGIiA5mc5/N3sBGwDLA3yQOlXjx\nSFw/SSAiYhSwucdmT2ATYAXgVolvSLwIysR20nYXDfW6mUAuImIUklgL+DLwHjjpYrjq9XDc2iAy\ni2hERJeQWAf+63I4tupSOrQkkOqgiIhRzOZ2mHXb4Ef2reVJQNKikqZLOr96v4KkSyXdKukSScu3\nOoaIiLGtv2UvB9eOksA+wM1AT73T/sClttcHflO9HxUkTao7ht4SU/M6Ma7E1JzENJi+lr1sTkuT\ngKTVgMnACUBPHdWOwMnV65OB97QyhhE2qe4A+jCp7gD6MKnuAPoxqe4A+jCp7gD6MKnuAPowqe4A\n+jCp7gB62DOmwlX7wOQh9w5arBUBNTgK2A9YtmHbS20/WL1+EFozCi4iopuURMBUSUPq7dOykoCk\n7YFZtqczvxSwAJeuSZ3ZPSkiogu0rIuopEOAXYBngXGU0sDZwObAJNsPSFoFuMz2C5ZbG2o2i4iI\nouPGCUjaGviC7R0kHQH82/bhkvYHlrc9ahqHIyLGknaOE+jJNocBb5d0K/DW6n1ERNSgY0cMR0RE\n63XciGFJJ0p6UNKNdcfSQ9Lqki6TdJOkGZL27oCYxkn6k6TrJd0s6dC6Y+rRe4Bg3STdKekvVUxX\n1x0PgKTlJZ0p6Zbq3+/1HRDTK6rfUc/Xox3yf/2A6m/vRkk/k7RkB8S0TxXPDEn71BTDC+6VwxmM\n23FJAPgx8M66g+hlLvA5268CXg/sKWnDOgOy/RTwFtubUKaYfYukN9UZU4PeAwTrZkpnhE1tb1F3\nMJWjgam2N6T8+91SczzY/lv1O9oUeC1ldatz6oxJ0prA7sBrbL8aWBT4cM0xTQR2o3Ry2RjYXtI6\nNYTS171yyINxOy4J2L6Css5mx7D9gO3rq9dzKH+wq9YbFdh+onq5BOWP46EawwH6HSDYCTomFknL\nAVvZPhHA9rO2H605rN7eBtxue2bNcTxGeQhbWtJiwNLAvfWGxAbAn2w/Zfs54HLgfe0Oop975ZAH\n43ZcEuh01ZPJpsCf6o0EJC0i6XrKoLvLbN9cd0zMHyA4r+5AGhj4taQ/S9q97mCAtYB/SvqxpOsk\nHS9p6bqD6uXDwM/qDsL2Q8B3gbuB+4BHbP+63qiYAWxVVb0sDbwLWK3mmHoMeTBuksAQSBoPnAns\nU5UIamV7XlUdtBrw5rrnMmlmgGBNtqyqOLajVOVtVXM8iwGvAb5v+zXA43TQHFqSlgB2AM7ogFjW\nAf4bWJNS+h4v6aN1xmT7r8DhwCXAhcB0OuuhB2h+MG6SQJMkLQ6cBfzU9rl1x9Ooqkq4ANis5lDe\nCOwo6Q7gNOCtkk6pOSZs3199/yeljrvudoF7gHtsX1O9P5OSFDrFdsC11e+rbpsBf7D9b9vPUgac\nvrHmmLB9ou3NbG8NPAL8re6YKg9KWhmgGow7a7ATkgSaIEnAj4Cbbf9v3fEASFqxp+Vf0lLA2ylP\nJLWxfaDt1W2vRalO+K3tj9cZk6SlJU2oXi8DvAOoteeZ7QeAmZLWrza9DbipxpB625mSxDvBX4HX\nS1qq+jt8G6XTQa0kvaT6vgbwXjqg6qxyHvCJ6vUngEEfWFs9gdyQSToN2Bp4saSZwNds/7jmsLYE\nPgb8RVLPjfYA20OesW8ErQKcLGkRSjL/ie3f1BhPXzqhd9BLgXPK/YPFgFNtX1JvSADsBZxaVb3c\nDnyy5niA5xPl2yg9cmpn+4aqNPlnSpXLdcAP640KgDMlvZjSaP1Z24+1O4CGe+WKPfdKyuDb0yX9\nB3AnsNOg18lgsYiI7pXqoIiILpYkEBHRxZIEIiK6WJJAREQXSxKIiOhiSQIREV0sSSDGJElzqu8v\nl7TzCF/7wF7vrxzJ60e0U5JAjFU9A2DWAj4ylBOr2SoHcsACH2RvOZTrR3SSJIEY6w6jzPg4vVoI\nZBFJ35Z0taQbJO0BIGmSpCsk/ZIySySSzq1mHp3RM/uopMOAparr/aTa1lPqUHXtG6tFbHZquPY0\nSWdUi8j8tCc4SYdVC6bcIOnbbf3NRNCB00ZEjLAvAV+wvQNAddN/xPYW1QpVv5fUM43EpsCrbN9V\nvf+k7YeruZmulnSm7f0l7VnNStqjp9TxPsoiIxsBKwHXSPpdtW8T4JXA/cCVkrakzIvzHtsbVLEt\n24KfP2JAKQnEWNd7Sut3AB+v5oD6I7ACsG617+qGBACwT7Vew1XA6sB6g3zWm4CfuZhFWWxkc0qS\nuNr2fdX0vtcDL6fMPvmUpB9Jei/w5LB/yohhShKIbvRfPcso2l6nYZGSx3sOqNZm2AZ4fbVmw3Rg\n3CDXNS9MOj2lhKcbtj0HLF6tSrUFZSrp7YE6JySMLpUkEGPdbGBCw/uLgc/2NP5KWr+fVb2WBR62\n/ZSkDShrS/eY20/j8RXAh6p2h5WANwNX088CO9WMncvbvhDYl1KVFNFWaROIsarnCfwG4LmqWufH\nwBTKKlXXVfPTz6LMB997FaaLgM9IupmyYMhVDft+SJlW/Frbu/ScZ/scSW+oPtPAfrZnSdqQF06r\nbUpy+qWkcZRE8bkR+ckjhiBTSUdEdLFUB0VEdLEkgYiILpYkEBHRxZIEIiK6WJJAREQXSxKIiOhi\nSQIREV0sSSAioov9P6uj7gbd7mgUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108a857f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ada = AdalineGD(epochs=10, eta=0.01).train(X, y)\n",
    "plt.plot(range(1, len(ada.cost_)+1), np.log10(ada.cost_), marker='o')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('log(Sum-squared-error)')\n",
    "plt.title('Adaline - Learning rate 0.01')\n",
    "plt.show()\n",
    "\n",
    "ada = AdalineGD(epochs=10, eta=0.0001).train(X, y)\n",
    "plt.plot(range(1, len(ada.cost_)+1), ada.cost_, marker='o')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Sum-squared-error')\n",
    "plt.title('Adaline - Learning rate 0.0001')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The two plots above nicely emphasize the importance of plotting learning curves by illustrating two most common problems with gradient descent:\n",
    "\n",
    "1. If the learning rate is too large, gradient descent will overshoot the minima and diverge.\n",
    "2. If the learning rate is too small, the algorithm will require too many epochs to converge and can become trapped in local minima more easily.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_learning_rate.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gradient descent is also a good example why feature scaling is important for many machine learning algorithms. \n",
    "It is not only easier to find an appropriate learning rate if the features are on the same scale, but it also often leads to faster convergence and can prevent the weights from becoming too small (numerical stability).\n",
    "\n",
    "A common way of feature scaling is standardization\n",
    "\n",
    "$$\\mathbf{x}_{j, std} = \\frac{\\mathbf{x}_j - \\mathbf{\\mu}_j}{\\mathbf{\\sigma}_j}$$\n",
    "\n",
    "where $\\mathbf{\\mu}_j$ is the sample mean of the feature $\\mathbf{x}_{j}$ and $\\mathbf{\\sigma}_j$ the standard deviation, respectively. After standardization, the features will have unit variance and are centered around mean zero. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# standardize features\n",
    "X_std = np.copy(X)\n",
    "X_std[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X_std[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEZCAYAAACU3p4jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8XfO9//HXO3MQBFFESKlQalZD8avSFrdUVVqKttre\nDnRQV93bcu+lc13DNVRbWi1VpbeGlqKqRU0tJaGIJqghhkSIEEOmk8/vj/XdsnNyztnrnLP2uN7P\nx2M/stf8WTvJZ6/9Wd/vdykiMDOzzjek2QGYmVljOOGbmZWEE76ZWUk44ZuZlYQTvplZSTjhm5mV\nhBO+5SLpAknfzLnuzZI+ld4fJun6+kbXXJIel7Rnen+8pB83Oyaznjjhl1xKznMljaixaqRXHm+s\nGxEXR8Teg4lxMCQdIulOSa9Imi3pr5KOLPgwb3wuEfGdiPj0YHcoaaKkpZJ6/T8q6SRJiyW9nF7T\nJZ0taZ3BHr9e0jlt1Ow4ysoJv8QkTQR2BJ4D3p9nk3rGUzRJxwJnACcDb4qINwGfA3bt7QuurwTb\nJH195gFcEhGrAmOBA4F1gHtaOenTZv+OOkmr/eO2xvoY8EfgIuDj1QskbStpSrpyvBQYVbVsrKTf\nSXou/Tq4WtL4ng4g6QhJt1ZNL5X0WUkzJL0o6fvd1v+kpGlpv7+XtMFATkzSasDXgSMj4oqIeBUg\nIu6NiMMjYlFa7wJJP5R0raRXgD0kvU/SVEkvSXpS0ond9v1RSU9Iel7S8d2WnSTpoqrpnSXdkc71\nXknvrFp2s6RvSLotfc7XS1ozLb4l/TlP0nxJO/V0mulFRHRFxDTgYGAOcGzVcfZLx35R0u2Stqxa\n9h+SnkrH/0dVaWpoKk89kpbdLWn9tGwzSTdIeiFt86Gq/V0g6Zz07+Pl9Itqo7Ssck73pXN6Yztr\nkIjwq6Qv4BHgMGATYBGwdpo/AngCOBoYChyUln8jLV+D7GpyFLAK8H/AlVX7vQn4ZHp/BHBr1bKl\nwFXAqsAEsl8Xe6dlBwAPA5uSXYycANw+wHPbB1gMDKmx3gXAPGCXND0SeCewRZreEpgFHJCmNwfm\nA7ulz+m0dJw90/ITgYvS+/HA88A+afrdaXrNNH1zOt+3pM/yJuC7admG6bPqNX7gpMqxus3/OvDX\n9H5bYDbwdrIvh48BjwHD0+f8JLBOWncDYKP0/jjg78AmVZ/DGsDKwEyyC4QhwDZkXzBvrfo8nwd2\nSP92fkH2K6T673+jZv/bL+vLV/glJWk3soR0VUQ8DEwDDk2LdwaGRcSZkV05Xg78rbJtRMyNiCsj\nYkFEvAJ8hyxJ5vW9iHg5ImaSJbmt0/zPkSW86RGxFPgusI2kCQM4xbWA59N+KudcudJ+LZ1/xW8i\n4i/p3BZGxJ8j4sE0fT9wadX5TQaujojbIvuV8F9kSeyNw1S9Pxy4NiJ+n/b1R+Bu4H1peQA/i4hH\nImIB2RfnNj3sp7+eJUvOAJ8Bzo2Iv0Xm58BCYBdgCdkX3BaShkfEkxHxz7Tdp4AT0r8NIuL+iJgL\n7Ac8FhEXRsTSiLgXuAKovlq/IiLujogu4OKqc7Imc8Ivr48Df4iI+Wn61ywr66wHPN1t/SdISUjS\nSpLOTa1TXgL+DKwmKW+SmlX1/jWyXwmQXdWemZLyi8ALaf4K5SJJP0plgfmSvtrDMV4A1qquyUfE\nOyJibFpWmR9kV6zV+95J0k2pZDUP+CxQKbWsBzxVtc/XquLsbkPgQ5XzSee0K1mdvafP4vWqz2Iw\nxlfFtCFwbLcY1gfWjYhHgS+T/VKYLekSSeum7SYAj/ZyTjt129+hwJvS8iD7RVH0OVkBhjU7AGs8\nSaOBDwNDJD2bZo8kS9pbAc+wYpLdkKwEBFl9eBKwY0Q8J2kbYArZF8Jghl99EvhmRFxSa8WI+BzZ\nL4Le/IXsSvYDZFeg/fFL4CyyUtMiSf/LsoT/DPDWyoqSVqpa1t2TZCWXz/Tz+JDvc1xhnfQFtz/w\nh6oYvh0R3+lxB9lnfYmkMcC5ZDe4P0b2JfgWsl9+1Z4E/hwR781zEtZafIVfTh8g+zn/VrJyytbp\n/W1k/9n/AiyR9CVJwyV9kKwGXLEK2ZXbS5LWIKtbD9QbNx6BHwHHS9ocshuvA72xFxHzyGrZP5B0\nkKQxkoakL6eVux2/u1WAF1Oy35FlpS6Ay4H9JFVa+nyD3v8f/QLYX9J7003QUZL20PI3uHv7VTSH\nrFS0cR+n+ca2koZJeitwCbA2cHpa9GPgc5J2VGbldFN6FUmTJO0paSTZl+MCoCtt9xPgm5Lekrbb\nKv1d/w6YJOnw9G9juKS3S9qsxvlUzK5xTlZHTvjl9DHgpxHxVEQ8l16zge+TJbcu4INkN1xfIPs1\ncHnV9mcAo8luzt0BXEfvV6Td2+93X6+6zf5vyK4wL02lovuBAbfhj4hTgH8D/p2sdDKL7Evl38m+\n1HqKD+Ao4BuSXiar0f+qap8PAp8n+xXwDDCX5UtC1efzFNmN6OPJbk4/SfbrSN3W72nb14BvA7en\n0smOPZ0icLCk+WQ3nn9L9kWxfUTMSvu5B/g02d/tXLKbxB9L248ku08yh6zuvxbwtbTsdLJ7Cn8A\nXiL74hiV7tm8FziErOz3bNpHpZlrT59n9fRJwIXpnCb3cE5WR4rwA1DMzMrAV/hmZiXhhG9mVhJO\n+GZmJeGEb2ZWEi3dDl+S7yibmQ1ARKzQRLalE34mPtvsCPpn3/3huqubHUXxOvG8OvGcwOfVbupx\nXjq3p7ku6ZiZlYQTvplZSTjhF27P6c2OoD468bw68ZzA59VuGndeLd3TNrtp2241fDOzZtO5bXrT\n1sxssF4bMnLk2ZttsME/3jZ06KLRUvs+ZjGC6Ooa8fqTT272wMKFX/wHrLS09lYZJ3wz63hrrvm1\n/7fvvhtO3Hff77686qrj5g4dOrTZIQ1YV1cXL788Z/h11/1yx+uu+9raL7xw5s15t3UN38w63rhx\nL2wwefJRc8eOXWdxOyd7gKFDhzJ27DqLJ08+au64cS/065nPTvhm1vGkGDJixKjWvWE5ACNGjAop\n+pXDnfDNzErCCd/MrCR809bMrIVMm3b7mPPOO2bzp56avtprr80fce21S39X1L59hW9m1kKGDx+1\ndNddD3rm058+/b6i9+0rfDOzPhz76c12X/Lqi5Vn9jJs5bGLTvvxP26t1/E22WT7VzfZZPtXH310\n6kpF79sJ38xKq3syhxUT+pJXXxwxZeXVF1Wmt+u2fjtxwjez0uqezKH/CT3Pl0arcMI3MxuEwX5p\nXHnl/46/8MITtgLYaKNtXjj99DvuKjrGCid8M7M+DFt57KLtutXwi9z/gQce8/SBBx7zdJH77I0T\nvpmVVvdkXplXPd2M0szrr78yZPHihUMAFix4bQjAqFH5B0nrTdMSvqQJwM+BtYEAzouIs5oVj5mV\nTxHJPM+XRn888cQDo488cqu9ACTxwQ+u8i9jx77ptYsvfvbGQcc62B0MwmLgmIi4V9IqwD2SboiI\nh5oYk5lZvxT9C2DDDd/2epGdrao1reNVRMyKiHvT+1eAh4D1mhWPmVmna4metpImAtsCdzY3EjOz\nztX0hJ/KOZcBR6crfTMzq4OmttKRNBy4HPhFRPym57X23X/Z+z2nw3EzGhGbmVn7OGUS3LhprbWa\n2UpHwPnAtIg4o/c1r7u6UTGZmbWn42YsfzGs/Xpaq5klnV2Bw4F3SZqaXvs0MR4zs47WtCv8iLiN\nFriHYGZWFk64ZmYl4YRvZlYSHkvHzKyJLrrovybecsv/TZg9+7Ex222399MnnXR14U+6qvAVvplZ\nLxYuRE8/zajK9Jw5jHj1VYYWeYy11pqwYPLk42bsvPMBM4vcb0+c8M2stGbPZkRX17LpWbMYWb38\ngQdY7dRT2frxxxn93HOMOPVUtvrrX1mzyBj23fczs/be+19nr7LKGoUOu9wTJ3wzK62LL2bj885j\nk64uuPVW1jz5ZLaZP3/ZFfz22zNvv/147NvfZocTTmCnt7+dWXvtxXOV5YsXo5/+lI3mzcvK47Nm\nMfLCC3nz0gENZByDP6EanPDNrLQ+9zlmzJnD6C9/mV0uu4xNjjySB8aMoat6nS23ZF7l/Q478EL1\nsqFDieHDWXrqqWw5YwYrn3YaW662GouGDCizamAn0Q9O+GZWWiutRNcuuzBrwQKGrb46CzbemFer\nlz/3HCNOO42t9t6bxw45hOlnnslWjz/O6MryIUPgsMN4fO21ee2UU9hum22Y84EPMMCnV/kK38ys\nbm69lTWvuYY3f/GL3Dd0KEsr5Z2KefMY/o538MzkyTy11148t99+PDZ3Lss97OS55xg5cyZjhg1j\n6UMPsUalvNN/vsI3M6ubmTNZ+aijeGCrrXj5K1/hwSFDiAULltXwJ03i1QMO4JnK9F578dx22/FS\nZXrxYnTGGbxt11159pxzuH3TTXnxzDPZoj81/CVLFvP6668MWbp0iZYu7dKCBa8NWbJkcVGnuBy3\nwzez0jr0UJ6svF9pJbo++1ke6c/2w4cTX/wiD44fzwLIyjvPPsuo/tTwf/SjL0269tpzJ1WmP/jB\nVdbfb78jpx911DkP9yeWPJzwzcwGoZLsIavpV0/n8YUv/HDGF77ww4YM++6SjplZSTjhm5mVhBO+\nWesbA0yomp6Q5pn1ixO+WetbHZgMbJBek9M8s37p9aatpLNzbP9SRPxngfGY2Ypmkj37+fA0/Ys0\nz3KK0NJFixZoxIhR9e/d1CCLFi1QhPo1iENfV/jvB+4G7kl/dn/dQ3alYWZWlLqUr+bMWfPJyy47\nZ425c58d3tXVVXuDFtbV1cXcuc8Ov+yyc9aYM2fNJ2tvsYwiev7Ck3RMRPxvnxvnWGcwJAXEZ+u1\nf7M2MYHs4uryNH0QcBmdeZVfp3N9bcjIkedsOmHCQ1sOG7ZwtNSAbq11EkEsWTLy9Zkz33r/woWf\nnw4r9XCVr3MjYoVz7DXhtwInfDMgu8JdnWVJbwIwD5jftIjqawOWL1/16yrWoLeEn7eGHyw/0ENE\nxJcKjM7Mejef5ZN7Pa7sO+lLpZPOpVB91fDvSa+RwHbADOBhYBtYfvAgM2t7rdISaAJZGecX6XUQ\ny9f082iVc2k5NUs6ku4EdouIxWl6OHBbROxU9+Bc0jFrpFYopRR1dd4K59JEPZd08rTDXx1YtWq6\n8hdiZsVwx6pl5rN8yWomyyd7f1aDkCfhfw+YIulCSRcCU4Dv1jcss1JphRJEEaWURsjzWbXLuTRc\nrlY6ktYFdkyTd0bErLpGtey4LulYWTS7BNFONzprfVbtdC51MsCSjqQhwLuBrSPit8AISTvW2MzM\nGmddYIeq6R3SvHZURMnGZaFe5Cnp/ADYBfhImn4lzTOzYgy2BDEeOIzsV/iO6f34fsbQCmWlPHG4\nFc8g5GmlMzUitq38mebdFxFb1z04l3SsHIooQewIfDS9vwi4awBxNLuslCcOt+LJZeCtdBZJeuMZ\nj5LGAf0asMfM+lSrBFHLGGDNquk1aU6JIk+pZLDllMF+VqWWJ+GfDVwJrC3pO8DtuJWOWSt5B9lg\nh9em1/vTvP5oVKmkESWbWkrbiidvK523AnulyT9FxEN1jWrZcV3SMattXWBrsh7xkDWdvg94th/7\naGSppBElm76UoBXPwFvp/BQYFRHfT6+HJJ1UjxDNbECeBaZVTU+jf8keWqdU0og4WuVcGy5PSWdv\n4EJJH6+ad0Cd4jErShFN7xpRk86jVrPLPCWKRnweeeIobTmlFeRJ+M8BuwMfkvSDNJaOWasrould\nETXpItRqdjmPbMz4J9PrsjSv6Dhr7SNPHHnWsTrpT7NMASeR1fLXi4iN6h6ca/g2OEU0vRtsTboo\nrdLsssObM3aKgTfLvBqyAfAj4kTgZODxYoMza4rS9rjsQaM+i1rH8d9JHdVM+BHx392mr46IPesX\nklkh8tSKi2gi2Iia9A5kPd0vSq+PsHxNP49acTZqULJaxyltL9hG6OuZtrdHxK6SXiF74lW1iIhV\ne9quXwfPWgC9D3guIrbsYblLOjZQeZveDbaJYCOa+K1LVrO/O03vADxN8c0uGzUoWa3juGw0aP18\nxGFE7Jr+XKWOUf2MrGPXz+t4DCunIh4LmGcfjXj84LMsn9zv7m1Fs770WtKRtEZfryIOHhG3Ai8W\nsS+zAShTE8EiyldFlFtqHadMfycN11dJ53GWPbx8A5Yl5rHAExHx5kICkCYCV7ukY01Qgh6Xyymi\nh+tgyy21jlO2v5M66X9JZyKApB8DV0bEtWl6X+DAOkXZg333X/Z+z+lw3IzGHds6XK1yTJ7k06j6\ner01ojSV5ziNiqPDnDIJbty01lp52uE/EBFvqzVvoHyFby1sAlnZ4vI0fRBZR6HqJLQDWUeoS9L0\nR4CL6V+dPc9xBquIYzQiTitEP6/wqzwj6T/Jfr4JOJTsCsas080kS27VJYzuye1usnth1Z2i+ntT\nNc9xBqvSw7Wy34H0cC1iH9ZEeTpeHQKsTTZE8hXp/Uf63CInSZcAdwCTJM2U9Iki9mtWkDHAm6qm\n30TPY+n0NRZ9q3QkKmLAsNIOOtYp+rzClzQMODsiDqvHwSOikC8OszrZlOwXbaVccygrlmsqY9Ff\nlabfT/aAoOvTdKVlS/cySHWirG6ZUr2OSyVWqD4TfkQskbShpJERsbBRQZm1iOnAL4H3pOlfpnnV\n/k5W6qyMRf/bNK8iT7nGpRJriDw1/MeA2yRdBbyW5kVEnF6/sMxawnxgdtX0bFYsYTwLDGdZwh/o\nWPRumWJ1l6eG/yhwTVp3FbL6owczsk5QxDjztdbZjKwUVFl+aJrXHx5wzApR8wo/Ik5qQBxmzVAZ\nZ75y4VNpUlm5Qs9Taqm1Tk/tnms/V3R5te4D5LlPYJarHf7awL8DmwOj0+xoxIiZbodvDVDEOPO1\nNGIceg84ZlUGPh7+xcA/gI3IHoDyOB68yeqvFcoURT0WsFbTTrOGyJPw14yInwCLIuLPEfEJwOPh\nW701Ylz0WuPMFxFDpWnnH9Pr0DSvPzzgmBUiTyudRenPWZL2A54hG0DNrJ4a0fv0aZZvV7+U5XuR\nFxFDnqadtdS6T+BmnZZLnoT/bUmrA8eSjV2/KnBMXaMya4xXWLHZ5SsFHyNP0848+/CAYzZoeVrp\nXJ3ezgP2qGs0Zss0ovdprdYtRcTgXrTWMvoaD//sqsnKuPiV90TEl+obmlvplFyjhgwuYoz4vrTC\n0MdWOv1vpXNPeo0k60U4A3gY2BYYUY8Qzao0YqCuWi1oPOCYdZS+HoByAYCkI4HdImJxmv4hcFtD\nojOrrzyDo5l1jDw3bVcnu1H7Qpqu/EQ1a3dFtKAxaxt5Ev73gCmSbk7T7yTrgGXW7mq1oHH93TpK\nnlY6P5P0e2Anshu2/xERs+oemVn91WpB4zFqrKPUHEsHQNJ4YCLZF0Sllc4tdY0Mt9KxustzBe8x\naqwNDfCZtpJOBg4mG+e7q2pR3RO+WZ25w5KVSp4a/oHApn7ilfVDp9S+83Sa6pRztRLI+wAUt7u3\n/mjEwGeNUBmj5sn06mmMmk45VyuBPFf4rwP3SvoTULnKj0b0tLW21YiBzxohT8mnU87VSiBPwr8q\nvar194k9ZmbWZHmaZV7QgDiss5RpwLAynau1uTyPOJwEfIcVH3G4UZ1jc7PM9lWmG5llOldrsnHM\nPm04i4fXWu8ZJowcULNM4GfAicDpZMMjfwIY2s84rXM4wS3PTTutKF9dnydqPqlsJhMvrbWOlt1T\nWk6ehD86Iv4oSRHxBHCSpCnAf+XY1jpPnt6n7qFqtszu6/PEIXlWnMnEmWTD2QzWgBP+AklDgUck\nfYHsEYcrFxCQtac8rVLccsVKYX2eOCfPel/h9HlHc9YJ9Y6nljwJ/2hgJeBLwDfJRs78eD2DMjOr\ns937WjiO2R8YyYJRtXayFs93TWWHtmminifhvzki/kb2c/wIAEkfBv5ax7isdeVpleKWK9ZMn6yx\nfO31eWLCWjzf1ddKU9nh8wXG1BLytNKZGhHb1ppXD26l05Ly3LT1jV2rh1qJnPV5YvuduGtBrfUu\n48MPAj8tJKoWJOjf4GmS9gX+BRgv6SyWPdN2DLC4LlFaO8jTKsUtV6w/vlprhUrrlVrJfHdua4la\neavqq6TzDNkzbQ9If1YS/svAMXWOy8w6wyfH8NLGvS0cw/wxQ+ga+hVO7z5GUXdO5AXo65m29wH3\nSbq46nm2awDrR8SLjQrQzFrS7mN4aZ9aK63GvNVTU8PezKeYZoiWQ56btjdIen9a9x5gjqTbI8JX\n+WYdaByzT6u1zkgWjNqJuxZcxofn1Fj1UTq4Vt5ucj3EPCJelvSvwM8j4kRJ99c7MOtovqnbBHm6\n5Q+hayjATCbek2OXTuRtJk/CHyppXeDDwH+meR4t0wbDPXGL98n1eGqbvlYYQtfQPN3ygVsLisla\nTJ6E/w3geuD2iLhL0sbAw/UNyzqce+Lm199u+bf3sYoTecnlGR7518Cvq6YfJbsiGzRJ+wBnkA3G\n9pOIOLmI/Zq1g7zd8lOt/Nh6x2Odr692+J+JiPP62jjPOn1sOxT4PvBu4Gngb5KuioiHBrI/aysd\n3RN3HLNP68Ru+db++rrC/6qk51nW/r5apPlfBgaU8IEdgUci4nEASZeStfl3wu98lWfFVhJ8T8+K\nbZY+x1gBdq01hG1K5JfiEoq1mL4S/i3A/jW2/8Mgjj2e5a/ongJ2GsT+rH00oydurm75tcZXAZjK\nDkUNYWvWUH11vDqizsfO2dJn36ovnT2nw3Ez6hOOtalciRzcLd861ykw6UbYtNZ6eVrp1MvTZLXc\niglkV/ndXHd1g+Kx1vNJYO3eFo5j9rhKB6C+duJEbp3uOJhxHLxxMSzYr6f1mpnw7wY2kTSRbNye\ng4GPNDEea6AxvPTtWuusxrzVa4yxsuBoznLrFbOcmpbwI2JJeoLW9WTNMs93C532lzeR5+yW73FW\nzApUM+FLGkXWbG5i1foREd8Y7MEj4jrgusHux+pvDC99e1SNpoaVpojulm/WmvJc4f+WrMncPUDN\nBwtY2/nqejy1Xq2VUrd8J3KzNpYn4Y+PiL3rHonVxXo8dVZlQKze5OiSD25Tbtb28iT8OyRtFRF/\nr3s0llueRA5vdMt3b04z63NohcoQyEOBT0h6DFiY5kVEbFXv4Moob7d8gJlM7LiHLJtZ/fR1hV/p\n8FQZRqGah0fuXW9d82t2ya9IQ9i6hGJmheqrp+3jAJIuioiPVi+TdBHw0Z6261C1xldhfZ44pFa3\nfHfJN7NmylPDf1v1hKRhwPb1CafhcnfLr5XMN+afHsLWzFpaXzX844GvAaMlVQ90tZiBj5DZKH12\nyYf83fIBJ3Iz6wiK6LscL+l7EfHVBsXT/dgB8dkeFvUZz/o8MaFGl3wAjuas3+NauZl1GMG5EbHC\n0PZ5Ev72rHiT9iXgiYhYUlyIPR47xjDv+ep5qzFv9bV4vusIft7r80+P5qxHcQcgMyup3hJ+nhr+\nOWQ1+0o7/C2BB4HVJB0ZEdcXF+aKXmb17ond46uYmQ1AnoT/DPCpiHgQQNLmwDeBfweuIBv8rJ6c\n3M3MCjAkxzqbVpI9QERMAzZLDzN3e3wzszaR5wr/QUk/BC4l64D1YWCapJFkLXbMzKwN5LnCPwJ4\nlOyB5UcD/wQ+Tpbs96xbZGZmVqiarXSaSVIE9NQs08zMejHgVjqSdgNOZMUHoGxUaIRmZlZXeWr4\n55OVc6YAfQ4vYGZmrStPwp+XHkVoZmZtLE/Cv0nSKWRt7ivj4RMRU+oWlZmZFS5Pwt+ZrL39Dt3m\nv6v4cMzMrF5qJvyI2KMBcZiZWZ3VbIcvaR1J50v6fZreXNKn6h+amZkVKU/HqwuAPwDrpemHgWPq\nFZCZmdVHnhr+WhHxK0lfBYiIxZLqOiyyNdemcHwXjKlMD4X50+E77XYMM1tenoT/iqQ1KxOSdiYb\nD986VBeMeSQbhhqAt1Ql5nY6hpktL0/CPxa4GthI0h3AOGByXaMyM7PC5Wmlc4+kdwKbplnTI8Kj\nZJqZtZm+HmJ+EFn7e1X9CTBJEhFxRQPisyYYCvPf0q2+3o7HMLPl9TpapqQL6OMBJxHxiTrFVB2D\nR8s0M+unfo+WGRFH1DUiK7VGtNIZB2cPq/o3vgSWzIEvFnkMcIsjax95btqaFa4RrXSGwbBn4Y0m\nxOvW6d+7WxxZu8jT8crMzDqAr/BLJk/5oYhSyFj40chlN/pZCPEifK4yPQ9W3RZWrZo2szrL20qn\nu3ArnfaUp/xQRClkJGhW1U3/dbr9OxoO/KJq3rv7aCAwUEtgybrdvriKPga4xZG1j77+I+9P3/8J\nnfBtwFaGl7eoSowr16HuXY8btD3xDVprF26lY2ZWErl+qkvaD9gcGFWZFxHfGOhBJX0IOAnYDHi7\nn55VnFo1+jzlh1qlkFr1+TRvuTLOQpb3AqyxDqzR23KofS+h1rm6WabZ8momfEnnAqOBPYEfAx8C\n7hzkce8HDgTOHeR+rJtaNfo8iahWUqxVnwdYE+b2FcdIoNY+at1LqHWubpZptrw8zTLfEREfA+ZG\nxNfJHnm4aY1t+hQR/4iIGYPZh5mZ9U+eK57X05+vSRoPvACsU7+QyquI0sALMHZ8VanktQG0fslT\nClnac+utN8yFsVtXxTG3hzi6auxjKQyZljXoyXXMgXA5xsokT8L/naSxwCnAPWnej2ttJOkGev5i\nOD4irs4b4L5ZayEA9oTpx0HH/jIoojQwEvRI1fSbB5Aka5VCFrLs8WeV6e6Ggy6pSvLv6hbHQmB8\njX0shvhI1XaLu31p1LofkadZZhGfuZtlWrOdApNuzFF5yZPw/yciFgCXS7qG7MbtglobRcR7cuy7\npuuysfgtJ8HS0VWJTXWoW9eqzwMMgaWbV8UxpFsco2Bprfr6GvDifYO4H+FmmVYWx8GM6othwX49\nrZcnGdwBbAeQEv8CSVMq8wpQ+M/0dpWnHDPYEkSjShiNKMfUkudc3ePXyqSvnrbrkv1yX0nSdiwb\nF39VYKVxPKmvAAALXklEQVTBHFTSgcBZwFrANZKmRsS+g9lnJ8hTjqlVgqhVxshTwqi1jzwljFrl\nmDzllsGWSvKcayN6/Jq1ir6u8N8LHEFWaj2tav584PjBHDQirgSuHMw+OlER5Zgiyhi19pHnF0Gt\nckyeOBtRKmlEj1+zVtFXT9sLgQslTY6IyxoYk3WAV2HVB2G1qunSXjm7JZC1ijxXkLdJOh8YHxH7\nSNoc2CUizq9zbKXTiDJHo1qULAYOr0ryzXgIcp5zbcTn4Y5Z1iryJPwLgJ8BJ6Tph4H/A5zwC9aI\nMkejrixXh5enNjnJ5TlXX2lbmeRJ+GtFxK8kfRUgIhZLqssws53OP+3NrJnyJPxXJK1ZmZC0M/BS\n/ULqXGX6ae/OSMv4s7BWkSfhH0vW+WkjSXcA44DJdY3K2p5/uSzjz8JaRc2EHxH3SPp/ZN12BUyP\niGbcgzMzs0HIMzzyaOAoYDeyVhe3Svph6nVr/dCIn/a+T2BmvclT0vk58DJZz1gBhwIXkY2Lb/3Q\niMRbpvsEZtY/eRL+FhGxedX0jZKm1SsgMzOrjzwPQJkiaZfKRGqlc08f65uZWQvKc4W/A3C7pJlk\nNfwNgOmS7gciIraqZ4DWP24CaGa9yZPw96l7FFYY36A1s97kaZb5eAPiMDOzOstTwzczsw7ghG9m\nVhJO+GZmJeGEb2ZWEk74ZmYl4YRvZlYSTvhmZiXhhG9mVhJO+GZmJeGEb2ZWEk74ZmYl4YRvZlYS\nTvhmZiXhhG9mVhJO+GZmJeGEb2ZWEk74ZmYl4YRvZlYSTvhmZiXhhG9mVhJO+GZmJeGEb2ZWEk74\nZmYl0ZSEL+kUSQ9Juk/SFZJWa0YcZmZl0qwr/D8AW0TE1sAM4GtNisPMrDSakvAj4oaIWJom7wTW\nb0YcZmZl0go1/E8C1zY7CDOzTjesXjuWdAOwTg+Ljo+Iq9M6JwCLIuKXve1nX9i/8n5PmH5cVgIy\nM7PkFJh0I2xaaz1FRCPiWfHA0hHAp4G9ImJBL+tEwGcbGpiZWZsTnBsR6j6/blf4fQYj7QMcB7yz\nt2RvZmbFalYN/2xgFeAGSVMl/aBJcZiZlUZTrvAjYpNmHNfMrMxaoZWOmZk1gBO+mVlJOOGbmZWE\nE76ZWUk44ZuZlYQTfsFOgUnNjqEeOvG8OvGcwOfVbhp5Xk74BcvTvbkddeJ5deI5gc+r3TTyvJzw\nzcxKwgnfzKwkmjZ4Wh6SWjc4M7MW1tPgaS2d8M3MrDgu6ZiZlYQTvplZSTjhF0zSKZIeknSfpCsk\nrdbsmIog6UOSHpTUJWm7ZsczWJL2kfQPSQ9L+o9mx1MEST+VNFvS/c2OpUiSJki6Kf37e0DSl5od\nUxEkjZJ0p6R7JU2T9N16H9MJv3h/ALaIiK3JHsf4tSbHU5T7gQOBW5odyGBJGgp8H9gH2Bz4iKS3\nNjeqQvyM7Jw6zWLgmIjYAtgZ+Hwn/H2lhz+9KyK2AbYC3iVpt3oe0wm/YBFxQ0QsTZN3Aus3M56i\nRMQ/IqJTnie8I/BIRDweEYuBS4EDmhzToEXErcCLzY6jaBExKyLuTe9fAR4C1mtuVMWIiNfS2xHA\nUGBuPY/nhF9fnwSubXYQtoLxwMyq6afSPGtxkiYC25JdTLU9SUMk3QvMBm6KiGn1PF5TnnjV7iTd\nAKzTw6LjI+LqtM4JwKKI+GVDgxuEPOfVIdwWuQ1JWgW4DDg6Xem3vVQN2Cbd67te0h4RcXO9jueE\nPwAR8Z6+lks6AvgXYK+GBFSQWufVQZ4GJlRNTyC7yrcWJWk4cDnwi4j4TbPjKVpEvCTpGmAH4OZ6\nHcclnYJJ2gc4Djgg3ZTpRCv04GszdwObSJooaQRwMHBVk2OyXkgScD4wLSLOaHY8RZG0lqTV0/vR\nwHuAqfU8phN+8c4GVgFukDRV0g+aHVARJB0oaSZZK4lrJF3X7JgGKiKWAF8ArgemAb+KiIeaG9Xg\nSboEuAOYJGmmpE80O6aC7AocTtaKZWp6dUJrpHWBG1MN/07g6oj4Uz0P6KEVzMxKwlf4ZmYl4YRv\nZlYSTvhmZiXhhG9mVhJO+GZmJeGEb2ZWEk741hIk7SFpheEbeptfwPEOqB5xUdLNkrbPEeNLkn5X\nY73ji4oz7W9QwwhIOknSsen91yUNugd4JSZJG6fhfecPdp9Wf074VlYHkg2NXJG3Q8otEbFfjXWK\nHhI7d2cZJb1tHxEnFtS5J9L+Hk3D+1obcMK3XCStLOmadDV3v6QPp/nbp6vjuyX9XtI6af7Nks5I\nvSLvl/T2NH9HSXdImiLpdkmT+hnDT9NDI6ZIen+af0R62Mx1kmZIOrlqm09Jmp62OU/S2ZJ2AfYH\nTkn72Sit/qG03vQ845JLWlfSLVXnuJuk7wGj07yL0nq/SZ/PA5I+XbX9K5K+lT7Tv0haO81/c5r+\nu6RvVa2/iqQ/SronLauc/8QU84Vkzy2YIOmENO9WYFNSgpZ0gaSD0t9bpdfq/ZKWpuUbp8/x7nRu\nm/YVk7WZiPDLr5ov4CDgvKrpVYHhZF3510zzDgbOT+9vAs5N73cH7k/vxwBD0/t3A5el93uQdS3v\nftw35gPfAQ5L71cHpgMrAUcAj6Z9jwQeJxvueD3gsbTuMLKHt5yVtv8Z8MGq49wEnJLe7wvc0Fcs\nafrfyEYSheziaZX0fn637camP0eTJeTK9FLgfen9ycAJ6f1VwOHp/VGV/ZGNlz4mvV8LeDi9nwh0\nATum6e2BvwOj0mfyMPBvPZ13mvc/wMnp/Z+At6T3OwF/6iumqn3M7/55+dV6L4+WaXn9HTg1XcH+\nLiJuk/Q2YAvgj6mKMBR4pmqbSyB7MIekVSWtCqwG/FzSW8iuOof3I4b3AvtL+kqaHglskPbzp4iY\nDyBpGlkSHAf8OSLmpfm/Bqp/UXQvfVyR/pyStq/lb8BPlY3k+JuIuK+X9Y6W9IH0fgKwCXAX2fDZ\n16T595ANngXwDrKSE8AvyL4MIPtS+a6k3cm+LNar/CoAnoiIu9L73YErIhu8b4Gk7gPDvXHekg4G\ntgPeo2z44V2AX1dVhUbUiMnaiBO+5RIRD0vaFngf8C1JfwKuBB6MiHf0Y1ffJEvOB0rakP4PBfvB\niHi4eoaknYCFVbO6yP5td69991rbTir7qGzfp/RFtjuwH3CBpNMj4qJuse1BNkz2zhGxQNJNZFfe\nkD26r2JpjmMeRnZlv11EdEl6rGpfr3Y7r+pz7XF00/SFfSKwe0SEpCHAvIjYtkYc1qZcw7dcJK0L\nLIiIi4FTyZ46NB0YJ2nntM5wSdU3Qg9O83cjSyQvk5WCKr8C+jua4/XAGw+wTl9A0HNCC7Ir8HdK\nWl3SMLKyVCXJz0+xDJikDYA5EfETsuF7K/EsTscjHePFlOw3IxtttJbbgUPS+8Oq5q8KPJeS/buA\nDXvZ/hbgA8oekj2G7AupWigblvcS4KMR8QJA+vt5TNLkdH6StFWNmKyNOOFbXlsCd0qaCvw38K3I\nngc7GThZ2RCvU8lKAhULJE0BfgB8Ks37H7KyxBSyElD1VXZPrVGiav43geHpxuEDwNd7WGfZhhHP\nkNX97wJuI6vnv5QWXwocl26AbtR9215i6W4P4N50Lh8CzkzzzwP+nm7a/h4YlspM3wX+0ssxqs/h\naLIHdf+d7D5EZf7FwA5p/kfJnu26wr4iYirwK+A+skds3sWK3k9WDvtJunE7Jc0/DPhU+vt8IK3X\nV0zWRjw8stVFKl0cGxFTaq5c3zhWjohX0xX3FWQ3lX87wH3tQXZO+xcZYyeQND8ixjQ7Duubr/Ct\n052UfpXcD/xzoMk+WQi8TTU6XpWJpI3Sr4FZzY7FavMVvplZSfgK38ysJJzwzcxKwgnfzKwknPDN\nzErCCd/MrCSc8M3MSuL/A7QF6NwC0qacAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10855acf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEPCAYAAACk43iMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHGW99vHvnQAmMYEYhRAlC4cQQcK+C+iwQwTElYMe\nRD0EFyABPWJAX+Vcx1eCvgoE9RwJi0EUV0TABBKBEVwIW1gCQoRDSFgyQdYECGT5vX9UDdOzd890\ndfV03Z/r6itd1d3V9wxM/7qe56nnUURgZmbFNCjvAGZmlh8XATOzAnMRMDMrMBcBM7MCcxEwMysw\nFwEzswLbKOs3kLQUeBlYD6yNiL0kjQJ+CYwHlgIfj4gXs85iZmbt1eJMIICmiNg1IvZK980AFkTE\nJOCmdNvMzGqsVs1B6rB9DDAnvT8HOLZGOczMrEStzgT+KOkuSVPTfaMjoiW93wKMrkEOMzPrIPM+\nAWC/iHhG0ubAAkkPlz4YESHJc1eYmeUg8yIQEc+k/z4r6XfAXkCLpC0jYoWkMcDKjq9zYTAz65uI\n6NgE3+OTM7sBw4AR6f23An8BDgO+A3w13T8DmNnFa6M6GY64ASI6346cV6Wf8Zwsf4dFyeiczlnv\ntwGUMyp5ftZ9AqOB2yTdCywEro+I+cBM4FBJS4CD0u2MLJ8FUx9tv++kx2DZRdm9p5nZwJBpc1BE\nPA7s0sX+54FDsnzvtvdaPFeaDEw5DSbvBS3L4O6vRSyeW4v3NzOrZ7XoGM5d+oE/V+IUYM8IqlkA\nmqt4rKw05x2gTM15ByhTc94BytScd4AyNecdoEzNeQfIgtI2pLojKaKSzo2yjsnWwO3AmAg2VPPY\nZmb1oNLPzkLNHRTB48BzwO55ZzEzqweFKgKpucCUvEOYmdUDFwEzswIrVJ9Aclw2Ibk4bVJE54vU\nzMwGMvcJ9CKCN0hmLj0i7yxmZnkrXBFIuUnIzIwCNgclx+adwGJgiwjWZfEeZmZ5cHNQGSJ4GngC\n2CfvLGZmeSpkEUi5ScjMCq/IReAPwAfyDmFmlqciF4GFwFYSW+UdxMwsL4UtAhGsB24Ejsw7i5lZ\nXgpbBFLuFzCzQivkENG292Bz4FGSoaKvZ/leZma14CGiFYjgWeAh4IC8s5iZ5aHQRSDlJiEzKywX\nARcBMyswFwFYBIyU2CbvIGZmtVb4IpAuMzkPDxU1swIqfBFI/QE3CZlZARV6iGjbe7EZsBzYMoJX\na/GeZmZZ8BDRPojgJeAe4MC8s5iZ1ZKLQBuPEjKzwnERaDMXmCJRkyYoM7N64CLQ5kGS38d2eQcx\nM6sVF4FUBEFyNuA1BsysMFwE2nO/gJkVioeItntP3gqsAN4Vwcu1fG8zs2rwENF+iOAV4K/AIXln\nMTOrBReBznz1sJkVhpuDOr0vE4FbSZqE6vOXY2bWDTcH9VMEjwKrgV3yzmJmljUXga55lJCZFYKL\nQNdcBMysEDIvApIGS1ok6bp0e5SkBZKWSJovaWTWGfrgT8BkibfnHcTMLEu1OBOYTrKYe2sn6wxg\nQURMAm5Kt+tKBK8DzcDhOUcxM8tUpkVA0lYkzSqXwJsTsx0DzEnvzwGOzTJDP7hJyMwaXtZnAucD\nXwE2lOwbHREt6f0WYHTGGfpqHnCExOC8g5iZZWWjrA4s6ShgZUQsktTU1XMiIiR1OxZf0jklm80R\n0VzVkD2IYJnEM8CewO21el8zs0qkn69NfX59VheLSfo2cAKwDhgCbApcTfKh2hQRKySNAW6JiE7T\nN+d1sVj7DMwE3ojgG3nmMDMrV91cLBYRZ0fE2IjYGvhX4OaIOAG4FjgxfdqJwDVZZagCTy1tZg2t\nltcJtJ5yzAQOlbQEOCjdrld/A/5FYkzeQczMsuC5g3rNwa+AeRFcnncWM7Pe1E1zUAPxUFEza1g+\nE+g1B6OBh4EtIlibdx4zs574TKDKImgBHgXem3cWM7NqcxEoj0cJmVlDchEoj/sFzKwhuQiU5y5g\nC4nxeQcxM6smF4EyRLAeuAE4Mu8sZmbV5CJQPjcJmVnD8RDRMkmMApaSDBVdk3McM7MueYhoRiJ4\nHrgfeH/eWczMqsVFoDJuEjKzhuIiUJk/4CJgZg3ERaAy9wPDJLbNO4iZWTW4CFQggsBXD5tZA3ER\nqJz7BcysYXiIaIUkRgBPA2MiWJ13HjOzUh4imrEIVgELSVZFMzMb0HosAkqMrVWYAcRNQmbWEMo5\nE5iXeYqBZy7wAYm6a64yM6tEj0Ugkg6DuyXtVaM8A8UjwFpgct5BzMz6o9eOYUmPABOBJ4BX0t0R\nETtlGqxOO4ZbSVddCze/G156BlatgeWzIhbPzTuXmRVbpZ+dG5XxnMPTf1urRd1+MNeKNHkKNO0B\ns8cAk5K9U7eRJuNCYGYDSa99AhGxFBgJHAMcDWyW7iuwsdPgB2Pa75s9Ecadlk8eM7O+6bUISJoO\nXAlsDowGrpQ0Letg9W3EkK73Dx9a2xxmZv1TTnPQScDeEfEKgKSZwO3ArCyD1bdV3awnsPq12uYw\nM+ufci8W29DN/YJaPgumPtp+30mPwbKL8sljZtY35ZwJXA4slHQ1SafwscBlmaaqcxGL50qTgSmn\nwbu2gs22gtunuVPYzAaaHoeIShoE7AusAfYnGSF0W0QsyjxYnQ8RbSUxmOS6gRMi+Fveecys2Cr9\n7CznOoF7I2KXfier0EApAgAS04D9Ijgu7yxmVmxZTCD3R0kflTQgPpBzcjlwiMS4vIOYmVWinDOB\n1cAwYD1JsxAkVwxvmmmwAXQmACDxPWB9BGfmncXMiquqzUGtfQIR8ZdqhKvEACwCE4C7gAleZ8DM\n8lLV5qCI2AD8sN+pCiCCpUAzcGK+SczMyuc+geq6AJguebEeMxsY3CdQRen6AncC50Rwfd55zKx4\nqj46KCKGR8SgiNg4Ikakt14LgKQhkhZKulfSQ5LOTfePkrRA0hJJ8yWNLDdsvYsggPOB0/POYmZW\njnImkBsk6QRJ30i3x5WzyExErAEOTK8x2Ak4UNL+wAxgQURMAm5KtxvJr4HtJTJdb8HMrBrKabv+\nEclVw59It1en+3oVEa+mdzcBBgMvkExJPSfdP4dkGoqGEcEbJJ3p0/POYmbWm3KKwN4R8UXgNYCI\neB7YuJyDp2cR9wItwC0R8SAwOiJa0qe0kExP3WguBj4ssUXeQczMelJOEXhD0uDWDUmbU+ZMohGx\nIW0O2gp4n6QDOzwetK1Y1jAi+CdJs9Dn885iZtaTcmYRvQj4HbCFpG8DHwW+XsmbRMRLkv4A7A60\nSNoyIlZIGgOs7O51ks4p2WyOiOZK3jdnFwA3SZwXwet5hzGzxiSpCWjq8+t7GyKavsn2wMHp5k0R\n8fcyXvMOYF1EvChpKHAj8J8kaxY/FxHnSZoBjIyITp3DA3GIaEcSNwI/j3izD8TMLFNVn0W0w8E/\nFxE/LvO5O5J0/A5Kbz+NiO9KGgX8ChgHLAU+HhEvdvH6RigCRwDnArulw0fNzDKVdRFYFBG79ilZ\nhRqkCAwCHgS+EEFzznHMrACymEq63fErfH6hRbABuBBfPGZmdarSM4GtIuLJDPOUvteAPxMAkBgG\nPAHsE8Fjeecxs8ZWteYgSV8u2QzazgICICK+39eQZQVrkCIAIHEuMCzCF5CZWbaq2Rw0AhhOMqzz\nC8A7gXeRjH3frT8hC+iHwAkSm+UdxMysVDmziN4GTImIVen2CGBuRByQabAGOhMAkPg5cFcEmZ5B\nmVmxZdExvAWwtmR7bbrPKnMBME0q6wI9M7OaKOcD6QrgDklXk/QLHAu++KlSEdwh8RTwQeC3eecx\nM4PyrxjeHdg/3bw1IhZlmorGaw4CkPgYMC2CTJvSzKy4srpOYBiwKiIuBJ6UtHWf0tnvgHESe+Qd\nxMwMyltU5hzgTNoWf9kEuDLDTA0rgnUkE/L54jEzqwvlnAl8iKQd+xWAiHiKZPio9c0lwBSJd+Yd\nxMysnCLwekS8uX6ApLdmmKfhRfAi8DPglLyzmJmVUwR+LenHwEhJJ5OsC3xJtrEa3ixgajqlhJlZ\nbnocHSRJwFhgO+CwdPeNEbEg82ANODqolMS1wPURXJx3FjNrHFWdSjotAg9ExORqhKtEAYrAQcAP\ngB281oCZVUtVh4imawDfLWmvfiezjm4hufr60LyDmFlxlTN30CPARJLpkF9Jd0dE7JRpsAY/EwCQ\n+Azw8QiOzDuLmTWGqq8sJmlCV/sjYmklwSpVkCIwhGSJzQMj6HXdZjOz3lT9iuGIWJp+4L8KbCi5\nWT9FsAb4H/A6A2aWj3LOBI4BvkeynsBKYDzw94jYIdNgBTgTAJAYDTwMTIzgubzzmNnAlsXcQd8C\n9gWWRMTWwMHAwj7msw4iaAGuAU7OO4uZFU85RWBtRPwTGCRpcETcAp4ArcouBE6R2DjvIGZWLOUU\ngRfS1cRuA34maRawOttYxRLBvcA/gI/mncXMiqWcPoHhwGskBeOTwKbAzyIi0/brovQJtJI4BuZ9\nF2Y9DiOGwKo1sHxWxOK5eWczs4Gj0s/OXlcWi4jWb/3rgZ/0MZf1atcNcPjWMG9S276p20iTcSEw\ns6yUs57Aakmr0tvrkjZIerkW4Yply1NhZoc+gdkTYdxp+eQxsyIo50xgeOt9SYOAY4B9sgxVTCOG\ndL1/+NDa5jCzIil3eUkAImJDRFwDHJFRngJbtabr/atfq20OMyuSXs8EJH2kZHMQsDtJR7FV1fJZ\nMHWbpAmo1UmPwbKL8stkZo2u1yIAHA1vTnW8jmSumw9mFaioIhbPlSYDU06Dt20G43eHYRe4U9jM\nstTrENG8FG2IaEcSnwK+DOwZwRt55zGzgSGLWUQvIjkTaD1ou/sRMa0vQXsN5iIg4DrgrgjOyTmO\nmQ0QWcwdNATYDVhCclXrrsAmwF3A3X0Jab1LVxs7GfiixC555zGzxlTOmcBCYP+IWJtubwz8OSL2\nzjRYwc8EWkmcCJwB7OVmITPrTRZnAiNJpopoNSLdZ7VxBfAkcFbeQcys8ZQzOmgmcI+kW0j6At4P\nbqOulQhC4nPAIonfp5PNmZlVRVmjgySNAfYm6RReGBEryjq4NJbkm+wW6WsvjohZkkYBvyRZoGYp\n8PGIeLHDa90cVELi08DpJKOF1uYcx8zqVNWbgyTtB6xKrxTeFDhT0vgyj78WOCNdhWwf4BRJ2wMz\ngAURMQm4Kd22ns0BnsLNQmZWReX0CfwP8KqknYEvAY+RfLvvVUSsiIh70/urgb8D7yKZf2hO+rQ5\nwLEV5i6cktFCp0rsnHceM2sM5RSBdRGxgeSD+ocR8UOSzuGKSJpAMrx0ITA6IlrSh1qA0ZUer4gi\neAr4KvATr0JmZtVQThFYJels4N+A6yUNhso+gNKFaX4LTI+IVaWPRdIpUZ+XLdennwDP4CY0M6uC\nckYHHQccD3w2IlZIGgd8t9w3SK8r+C3w07RfAaBF0pbp8cYAK7t57Tklm80R0Vzu+zaqdLTQybSN\nFro/70xmlh9JTUBTn19fydxBko6KiOsreL5I2vyfi4gzSvZ/J913nqQZwMiImNHhtR4d1AOJzwKn\nAnt7tJCZtar63EEdDr4oInat4Pn7A7cC99PW5HMWcAfwK2AcHiLaJ+ncQnOBv0bwX3nnMbP6UFdF\noD9cBHonMRa4BzjYzUJmBtlMG1HqcxU+3zIUwXKSDuLLPVrIzPqinAnkNgI+AEygrSM5IuL7mQbz\nmUBZ0mahecCfI/hW3nnMLF+VfnaWMzroOpLlJB8ANvQ1mGUjHS00FbgnHS30QN6ZzGzgKOdM4P6I\n2KlGeUrf12cCFZA4CfgCsI9HC5kVVxZ9AvMlHd6PTFYblwL/BM7MO4iZDRzlnAl8GLiSpGC0fsOM\niNi0+1dVIZjPBComMY5ktbcDI1icdx4zq70s1hheSjLh2+J0DqGacBHom7R/4HMkzULr8s5jZrWV\nRXPQMuDBWhYA65dLgOdxs5CZlaGcM4E5wNYkwxBb17j1ENE65mYhs+LKYojo4+ltk/QmPOtnXYtg\nmcTXSC4i29fNQmbWnYqmjaglnwn0T3IR2XV3w7XD4aWnYdUaWD4rYvHcvLOZWXaqfiaQLjDfUUTE\nQRUlsxqbfCTsPwpmjwe2TfZN3UaajAuBmbUqp09gj5LNIcBHSFYb+0qmwXwm0C/SkTfAvC6u75hy\nQ8TcI2ufyMxqoepnAhFxV4ddf5Z0Z8XJrMZGDOl6//Chtc1hZvWsnOagUSWbg4A9gEwvFLNqWLWm\n6/2vvdH1fjMronJGB91D22igdSSLwPx7VoGsWpbPgqnbwOyJbfu+9DJ8/p0SIyJY1f1rzawouu0T\nkLQXsDwinkm3P03SH7AU+GZEPJ9pMPcJ9Js0eQqMOy1pAlr9Gjz5A7j/WGB74EgXArPGU7VpIyQt\nAg6OiOclvQ/4JcmatrsC20XER6sRuNtgLgKZkBgE/DcwmaQQvJxzJDOrompOGzGo5Nv+ccCPI+K3\nEfF13hxyaANNBBtIppy+H7hRYrOcI5lZjnoqAoMltS5ZeAhQer1AOX0JVqfSQvBFkv4eFwKzAuup\nCFwF/EnStcCrwG0AkrYFXqxBNstQBEHSvHcnMF9iZM6RzCwHPV4sJmlfYEtgfkS8ku6bBAyPiHsy\nDeY+gZpI1yi+AHgvcFgEL+Qcycz6oerrCeTFRaB20kJwPrA/cKgLgdnAlcV6Atbg0qahM4BbgT9K\njOrlJWbWIFwEDHizEHyZZADATRJvzzmSmdWAi4C9KS0EXwHmkxSCd+Qcycwy5iJg7aSFYAbJSnIu\nBGYNzuP9rZMIQuJsYANws8TBETybdy4zqz4XAetSWgi+TlIIbpE4KIKVeecys+pyEbBupYXgG7Qv\nBC155zKz6nERsB6lfQTflNoVghV55zKz6nARsLJE8J8SAfPvkH70GGwiL15vNvC5CFgFJt8Fh06H\na5ra9nnxerOBzENErQJjp8H5Ha4mnj0xWbjGzAYiFwGrQHeL1286vLY5zKxaXASsAt0tXj9hd4lD\napvFzKoh0yIg6TJJLZIeKNk3StICSUskzZfkeewHjOWzYOqj7fed9BgMmwlcKjHbC9SYDSyZTiUt\n6QBgNXBFROyY7vsO8M+I+I6krwJvi4gZXbzWU0nXoc6L1y+7KGLxXIlNgfOAo4AvRHB9zlHNCqnu\n1hOQNAG4rqQIPAy8PyJaJG0JNEfEdl28zkVgAJJoAi4BbgdOj+Cf+SYyK5aBsJ7A6Ihoveq0BRid\nQwbLSATNwM7ASuABiY+li9aYWR3K9TqBiAhJ3Z6KSDqnZLM5IpozD2X9FsErwJckfg1cChwv8UVf\naWxWfZKagKY+vz6n5qCmiFghaQxwi5uDGpfEEOD/AFNJ1iq4Ip2KwswyMBCag64FTkzvnwhck0MG\nq5EI1kTwNeBwkiUs50qMyzmWmaWyHiJ6FfBX4N2Slkv6DDATOFTSEuCgdNsaXASLgD2BPwN3S3xe\n8nUqZnnLvDmor9wc1Lgk3gNcBqwBToLJk5IpKUYM8aR0Zv1T6WenJ5CzmovgIYn9gOlw891wyBtw\nQckylp6UzqxWfCZguZI+fCtcfUDnR6bcEDH3yNonMhvYBkLHsFmJjTZ0vX/40NrmMCsmFwHLWbeT\n0u0mMVViWG3zmBWLi4DlrLtJ6YbOBI4GnpCYKTE2j3Rmjc59Apa77ialSx5jInAK8CngZuBC4C++\n4Mysa3U3gVxfuQhYKYkRJBcXTgNWAbOAX0Tweq7BzOqMi4A1tPQCsyNIisEuwMXAf0fwTK7BzOqE\ni4AVhsT2wKnAJ4C5wIUR3JE8NnmKL0CzInIRsMKRGAn8O0lBWAHfuxUe+QhcvE3bs6Y+Cn+b7kJg\njc5FwApLYjBwNHzpUvj+qM7P8AVo1vh8sZgVVgTrI7gGnnyg62eM3sKT1pm15z8Ia0DdXYC2+SSg\nReJKiU9KvKPr55kVh4uANaDuLkCbexywB3Ab8DHgMYmFEudI7J02J5kVivsErCH1dAFa23PYBNgf\nODK9jQbmA/OAGyN4tvMxPeLI6ps7hs36KF3x7AiSgnAQsISkIMyDXd8Oe5wPsye2vcIjjqz+uAiY\nVUF6lrAfb54lnPVuOHfjzs/0iCOrLx4dZFYFEbwRwS0RnBnBjrDs7q6fue0OEsdLbO8+BRuIvLKY\nWVmef6nr/WvWAB8B/gvYUmIxcG/J7f4IXu3uqO5nsLy5CJiVZfksmLpN+z6Bkx6D20+PoHXG002B\nnUnmNNoTmApsL/EE7QvDoghWJgVg3ws79DN4aU2rKfcJmJWpnBFHnV/DxsD2JIWh9LYGTt+4/drK\nrdzPYH3njmGzOichYCxM/QPMntz5GWethnNvBJaW3J4Alkawqvfju4mpyCr97HRzkFmNpQviLJOe\nfAroogg8/gDwK2AC8B5gSnp/vMQaSooCHYoETN7PTUxWCRcBs9x018+w+Fut/Qyl0jOId5AUhAnA\neGAScFjbvqPfAud2+LuePRFO/L8SrwHPprfnIlhXblKfXTQuFwGznEQsnitNBqaU1c+QnkG0fojf\n2fHxpEgsvY3k+oYOttgKOAfYPL29TeKlkuP1cDtuR3jvNztMzd3vs4ssCouLVeVcBMxylH5AVeVD\nKoKQXlzd9aMP3hXBm53N6TUNo2grCqW3icC+bdtbT4KZb2l/vNkT4Yw5EjcBL6e3VSX3X+5m/6oI\n1mcxMiqr0VaNXqxcBMwaSndNTMsuKn1WBOtp+7bfI+l/m4H3d35k1TPANcCmJbfx6b8jOuxvvQ1P\nmqWO2qjrwjLtEom5wKvAax1uXe0r2T/xzPY/d+sxp5xGHwvtQCpWfeUiYNZAKm1iKk93U3M//VQE\nv6jkSOl6DsNg+XySs40O3ngJWJg8h6Hp7e0l91tvwzrvmzy263fd4xCJFuCNym/vPwx+OK798WZP\nhJPPl9gBWFf5bcdvVrtY9YeLgFmDqWYTU6K8s4vysrEBWC09/3LXz1i2NILZfUkp3X0DcHjnR+79\nE8k61Jt0uL2li30dbtHFGRDA0KEks85uBAxO/y3zNv7dXR9z+NCKfuAqcREwsx5lc3ZRvcLS+zH/\n9/sRrOzLEaXHPwJs2/mRfzwYwX/07Zj3dVOsVr/Wl+P1ly8WM7Nc9OUK7Fofs+v2+5Meg9unVbdP\noH/HbH98XzFsZlY1A6FYtT+2i4CZWWF5PQEzMyubi4CZWYHlVgQkHSHpYUn/kPTVvHKYmRVZLkVA\n0mDgBySLer8HOF7S9nlk6S9JTXln6M1AyAjOWW3OWV0DJWel8joT2At4NCKWRsRa4BfAB3PK0l9N\neQcoQ1PeAcrUlHeAMjXlHaBMTXkHKFNT3gHK1JR3gCzkVQTeBSwv2X4y3WdmZjWUVxGoz3GpZmYF\nk8t1ApL2Ac6JiCPS7bOADRFxXslzXCjMzPqg7i8Wk7QR8AhwMPA0cAdwfET8veZhzMwKLJcJ5CJi\nnaRTgRtJZuC71AXAzKz26nbaCDMzy17dXTE8EC4ikzRW0i2SHpS0WNK0vDP1RNJgSYskXZd3lu5I\nGinpN5L+LumhtN+o7kg6K/3v/oCkn0t6S++vyp6kyyS1SHqgZN8oSQskLZE0X9LIPDOmmbrK+d30\nv/t9kq6WtFm9ZSx57MuSNkgalUe2Dlm6zCnptPT3uVjSed29vlVdFYEBdBHZWuCMiNgB2Ac4pU5z\ntpoOPER9j8q6EJgbEdsDOwF11zwoaQIwFdgtInYkacr81zwzlbic5O+m1AxgQURMAm5Kt/PWVc75\nwA4RsTOwBDir5qna6yojksYChwJP1DxR1zrllHQgcAywU0RMBv5fbwepqyLAALmILCJWRMS96f3V\nJB9Y78w3VdckbQVMAS4B6nJW1vSb3wERcRkkfUYR8VLOsbryMskXgGHp4IZhwFP5RkpExG3ACx12\nHwPMSe/PAY6taagudJUzIhZExIZ0cyGwVc2Dtc/T1e8S4PvAmTWO061ucn4BODf9/CQiel1Dut6K\nwIC7iCz9drgryf+89eh84CvAht6emKOtgWclXS7pHkmzJQ3LO1RHEfE88D1gGcmothcj4o/5purR\n6IhoSe+3kCyHWO8+Sw7r7PZG0geBJyPi/ryz9GJb4H2SbpfULGmP3l5Qb0WgnpsrOpE0HPgNMD09\nI6grko4CVkbEIur0LCC1EbAb8KOI2A14hfpoumhH0jbA6cAEkjO/4ZI+mWuoMkUyAqSu/74kfQ14\nIyJ+nneWUukXkrOBb5buzilObzYC3hYR+5B8+ftVby+otyLwFDC2ZHssydlA3ZG0MfBb4MqIuCbv\nPN14L3CMpMeBq4CDJF2Rc6auPEnyLevOdPs3JEWh3uwB/DUinouIdcDVJL/jetUiaUsASWOgb+vs\n1oKkT5M0W9ZjUd2GpPDfl/4tbQXcLWmLXFN17UmS/y9J/542SHp7Ty+otyJwF7CtpAmSNgGOA67N\nOVMnkgRcCjwUERfknac7EXF2RIyNiK1JOjBvjohP5Z2ro4hYASyXNCnddQjwYI6RuvMwsI+koen/\nA4eQdLjXq2uBE9P7JwJ1+WVF0hEk31o/GBFr8s7TUUQ8EBGjI2Lr9G/pSZLBAfVYVK8BDgJI/542\niYjnenpBXRWB9NtV60VkDwG/rNOLyPYD/g04MB16uSj9H7ne1XNzwGnAzyTdRzI66Ns55+kkIu4D\nriD5stLaNnxxfonaSLoK+CvwbknLJX0GmAkcKmkJyQfDzDwzQpc5PwtcBAwHFqR/Sz+qk4yTSn6X\nperi76ibnJcB/5IOG70K6PVLny8WMzMrsLo6EzAzs9pyETAzKzAXATOzAnMRMDMrMBcBM7MCcxEw\nMyswFwFrSJJWp/+Ol3R8lY99doftv1Tz+Ga15CJgjar1ApitgU9U8sJ0htCetJvqOCL2q+T4ZvXE\nRcAa3UzggPRK1OmSBqWLmNyRLmJyMoCkJkm3Sfo9sDjdd42ku9LFOaam+2YCQ9Pj/TTd13rWofTY\nD0i6X9LHS47dLOnX6WIfV7aGkzRTySI190n6bk1/M2bktMawWQ19FfiPiDgaIP3QfzEi9lKyKtif\nJc1Pn7uB8TbhAAABq0lEQVQryeImrYuGfCYiXpA0FLhD0m8iYoakUyJi15L3aD3r+DCwM8m0F5sD\nd0q6NX1sF5KFkp4B/iJpP5K5iI6NiO3SbJtm8POb9chnAtboOk75exjwKUmLgNuBUcDE9LE7SgoA\nwHRJ9wJ/I5nRdtte3mt/4OeRWAn8CdiTpEjcERFPp1M63wuMB14E1ki6VNKHgNf6/FOa9ZGLgBXR\nqRGxa3rbpmRhmFdanyCpCTgY2CcidgEWAUN6OW7Quei0niW8XrJvPbBxRKwnWU3vN8BRwA19+WHM\n+sNFwBrdKmBEyfaNwBdbO38lTepmFbNNgRciYo2k7UjWkm61tpvO49uA49J+h82B9wF30M0CJJLe\nCoyMiHnAl0iaksxqyn0C1qhav4HfB6xPm3UuB2aRLBByT7omwErgQ+nzS6fUvQH4vKSHgEdImoRa\nXQzcL+nuiDih9XUR8TtJ+6bvGcBXImKlpO3pPP1wkBSn30saQlIozqjKT25WAU8lbWZWYG4OMjMr\nMBcBM7MCcxEwMyswFwEzswJzETAzKzAXATOzAnMRMDMrMBcBM7MC+/+tFfzpda+57gAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108b13748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.evaluate import plot_decision_regions\n",
    "\n",
    "ada = AdalineGD(epochs=15, eta=0.01)\n",
    "\n",
    "ada.train(X_std, y)\n",
    "plot_decision_regions(X_std, y, clf=ada)\n",
    "plt.title('Adaline - Gradient Descent')\n",
    "plt.xlabel('sepal length [standardized]')\n",
    "plt.ylabel('petal length [standardized]')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(1, len( ada.cost_)+1), ada.cost_, marker='o')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Sum-squared-error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_animation.gif' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Online Learning via Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The previous section was all about \"batch\" gradient descent learning. The \"batch\" updates refers to the fact that the cost function is minimized based on the complete training data set. If we think back to the perceptron rule, we remember that it performed the weight update incrementally after each individual training sample. This approach is also called \"online\" learning, and in fact, this is also how Adaline was first described by Bernard Widrow et al. [[3](#References)]\n",
    "\n",
    "The process of incrementally updating the weights is also called \"stochastic\" gradient descent since it approximates the minimization of the cost function. Although the stochastic gradient descent approach might sound inferior to gradient descent due its \"stochastic\" nature and the \"approximated\" direction (gradient), it can have certain advantages in practice. Often, stochastic gradient descent converges much faster than gradient descent since the updates are applied immediately after each training sample; stochastic gradient descent is computationally more efficient, especially for very large datasets. Another advantage of online learning is that the classifier can be immediately updated as new training data arrives, e.g., in web applications, and old training data can be discarded if storage is an issue. In large-scale machine learning systems, it is also common practice to use so-called \"mini-batches\", a compromise with smoother convergence than stochastic gradient descent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the interests of completeness let us also implement the stochastic gradient descent Adaline and confirm that it converges on the linearly separable iris dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class AdalineSGD(object):\n",
    "    \n",
    "    def __init__(self, eta=0.01, epochs=50):\n",
    "        self.eta = eta\n",
    "        self.epochs = epochs\n",
    "\n",
    "    def train(self, X, y, reinitialize_weights=True):\n",
    "\n",
    "        if reinitialize_weights:\n",
    "            self.w_ = np.zeros(1 + X.shape[1])\n",
    "        self.cost_ = []\n",
    "\n",
    "        for i in range(self.epochs):\n",
    "            for xi, target in zip(X, y):\n",
    "                output = self.net_input(xi)\n",
    "                error = (target - output)\n",
    "                self.w_[1:] += self.eta * xi.dot(error)\n",
    "                self.w_[0] += self.eta * error\n",
    "            \n",
    "            cost = ((y - self.activation(X))**2).sum() / 2.0\n",
    "            self.cost_.append(cost)\n",
    "        return self\n",
    "\n",
    "    def net_input(self, X):\n",
    "        return np.dot(X, self.w_[1:]) + self.w_[0]\n",
    "\n",
    "    def activation(self, X):\n",
    "        return self.net_input(X)\n",
    "\n",
    "    def predict(self, X):\n",
    "        return np.where(self.activation(X) >= 0.0, 1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One more advice before we let the adaline learn via stochastic gradient descent is to shuffle the training dataset to iterate over the training samples in random order. \n",
    "\n",
    "We shall note that the \"standard\" stochastic gradient descent algorithm uses sampling \"with replacement,\" which means that at each iteration, a training sample is chosen randomly from the entire training set. In contrast, sampling \"without replacement,\" which means that each training sample is evaluated exactly once in every epoch, is not only easier to implement but also shows a better performance in empirical comparisons. A more detailed discussion about this topic can be found in Benjamin Recht and Christopher Re's paper *Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences* [[4](#References)].\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEZCAYAAACU3p4jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8XfO9//HXO3MQBFFESKlQalZD8avSFrdUVXSgg+pt\nlQ7qqnuVey8dtHUNt6Xa0lKqSm8NbRVVLWpqqcQcEtQQQxJEIoaT4eTz+2N9t+yc7HP2Oufseb2f\nj8d+ZK/5s3aSz/7uz/qu71JEYGZmnW9IswMwM7PGcMI3MysIJ3wzs4JwwjczKwgnfDOzgnDCNzMr\nCCd8y0XShZK+lXPdmyV9Nr0/VNL19Y2uuSQ9KWnP9P4EST9tdkxmlTjhF1xKznMljaiyaqRXHm+u\nGxGXRMTeg4lxMCR9TNKdkl6VNFvS3yUdWePDvPm5RMR3IuJzg92hpImSlkrq9f+opJMlLZb0SnpN\nl3S2pHUGe/x6See0UbPjKCon/AKTNBHYEZgDfDDPJvWMp9YkHQt8HzgVeEtEvAX4ArBrb19wfSXY\nJunrMw/g0ohYFRgLHAisA0xp5aRPm/076iSt9o/bGutTwJ+Bi4FPly+QtK2kqanleBkwqmzZWEl/\nkDQn/Tq4WtL4SgeQdJikW8uml0o6QtIMSS9L+mGP9Q+XNC3t94+SNhjIiUlaDfgGcGREXBkRrwFE\nxL0R8YmIWJTWu1DSjyVdK+lVYA9JH5B0j6T5kp6WdFKPfX9S0lOSXpR0Qo9lJ0u6uGx6Z0l3pHO9\nV9K7y5bdLOmbkm5Ln/P1ktZMi29Jf86TtEDSTpVOM72IiO6ImAZ8FHgBOLbsOPulY78s6XZJW5Yt\n+w9Jz6TjP1JWmhqaylOPpWV3S1o/LdtM0g2SXkrbHFy2vwslnZP+fbySflFtlJaVzum+dE5vbmcN\nEhF+FfQFPAYcCmwCLALWTvNHAE8BRwNDgYPS8m+m5WuQtSZHAasA/wdcVbbfm4DD0/vDgFvLli0F\nfg+sCkwg+3Wxd1p2APAosClZY+RE4PYBnts+wGJgSJX1LgTmAbuk6ZHAu4Et0vSWwCzggDS9ObAA\n2C19Tmek4+yZlp8EXJzejwdeBPZJ0+9N02um6ZvT+b4tfZY3Ad9NyzZMn1Wv8QMnl47VY/43gL+n\n99sCs4F3kn05fAp4AhiePuengXXSuhsAG6X3xwH3A5uUfQ5rACsDM8kaCEOAbci+YN5e9nm+COyQ\n/u38kuxXSPnf/0bN/rdf1Jdb+AUlaTeyhPT7iHgUmAYckhbvDAyLiB9E1nK8AvhHaduImBsRV0VE\nV0S8CnyHLEnm9b2IeCUiZpIlua3T/C+QJbzpEbEU+C6wjaQJAzjFtYAX035K51xqab+ezr/ktxHx\nt3RuCyPirxHxUJp+ALis7PwmA1dHxG2R/Ur4L7Ik9uZhyt5/Arg2Iv6Y9vVn4G7gA2l5AD+PiMci\noovsi3ObCvvpr+fJkjPA54FzI+IfkfkFsBDYBVhC9gW3haThEfF0RPwzbfdZ4MT0b4OIeCAi5gL7\nAU9ExEURsTQi7gWuBMpb61dGxN0R0Q1cUnZO1mRO+MX1aeBPEbEgTf+GZWWd9YBne6z/FCkJSVpJ\n0rmpd8p84K/AapLyJqlZZe9fJ/uVAFmr9gcpKb8MvJTmr1AukvSTVBZYIOn4Csd4CVirvCYfEe+K\niLFpWWl+kLVYy/e9k6SbUslqHnAEUCq1rAc8U7bP18vi7GlD4ODS+aRz2pWszl7ps3ij7LMYjPFl\nMW0IHNsjhvWBdSPiceCrZL8UZku6VNK6absJwOO9nNNOPfZ3CPCWtDzIflHU+pysBoY1OwBrPEmj\ngY8AQyQ9n2aPJEvaWwHPsWKS3ZCsBARZfXgSsGNEzJG0DTCV7AthMMOvPg18KyIurbZiRHyB7BdB\nb/5G1pL9EFkLtD9+BZxFVmpaJOl/WZbwnwPeXlpR0kply3p6mqzk8vl+Hh/yfY4rrJO+4PYH/lQW\nwykR8Z2KO8g+60sljQHOJbvA/SmyL8G3kf3yK/c08NeIeH+ek7DW4hZ+MX2I7Of828nKKVun97eR\n/Wf/G7BE0lckDZf0YbIacMkqZC23+ZLWIKtbD9SbFx6BnwAnSNocsguvA72wFxHzyGrZP5J0kKQx\nkoakL6eVexy/p1WAl1Oy35FlpS6AK4D9JJV6+nyT3v8f/RLYX9L700XQUZL20PIXuHv7VfQCWalo\n4z5O881tJQ2T9HbgUmBt4My06KfAFyTtqMzK6aL0KpImSdpT0kiyL8cuoDtt9zPgW5LelrbbKv1d\n/wGYJOkT6d/GcEnvlLRZlfMpmV3lnKyOnPCL6VPABRHxTETMSa/ZwA/Jkls38GGyC64vkf0auKJs\n++8Do8kuzt0BXEfvLdKe/fd7rlfeZ/+3ZC3My1Kp6AFgwH34I+I04N+Afycrncwi+1L5d7IvtUrx\nARwFfFPSK2Q1+l+X7fMh4ItkvwKeA+ayfEmo/HyeIbsQfQLZxemnyX4dqcf6lbZ9HTgFuD2VTnas\ndIrARyUtILvw/DuyL4rtI2JW2s8U4HNkf7dzyS4SfyptP5LsOskLZHX/tYCvp2Vnkl1T+BMwn+yL\nY1S6ZvN+4GNkZb/n0z5K3VwrfZ7l0ycDF6VzmlzhnKyOFOEHoJiZFYFb+GZmBeGEb2ZWEE74ZmYF\n4YRvZlYQLd0PX5KvKJuZDUBErNBFtqUTfiaOaHYE/bPv/nDd1c2OovY68bw68ZzA59Vu6nFeOrfS\nXJd0zMwKwgnfzKwgnPBrbs/pzY6gPjrxvDrxnMDn1W4ad14tfadtdtG23Wr4ZmbNpnPb9KKtmdlg\nvT5k5MizN9tgg0feMXTootFS+z5mMYLo7h7xxtNPb/bgwoVffgRWWlp9q4wTvpl1vDXX/Pr/23ff\nDSfuu+93X1l11XFzhw4d2uyQBqy7u5tXXnlh+HXX/WrH6677+tovvfSDm/Nu6xq+mXW8ceNe2mDy\n5KPmjh27zuJ2TvYAQ4cOZezYdRZPnnzU3HHjXurXM5+d8M2s40kxZMSIUa17wXIARowYFVL0K4c7\n4ZuZFYQTvplZQfiirZlZC5k27fYx5513zObPPDN9tddfXzDi2muX/qFW+3YL38yshQwfPmrprrse\n9NznPnfmfbXet1v4ZmZ9OPZzm+2+5LWXS8/sZdjKYxed8dNHbq3X8TbZZPvXNtlk+9cef/yelWq9\nbyd8MyusnskcVkzoS157ecTUlVdfVJrersf67cQJ38wKq2cyh/4n9DxfGq3CCd/MbBAG+6Vx1VX/\nO/6ii07cCmCjjbZ56cwz77ir1jGWOOGbmfVh2MpjF23Xo4Zfy/0feOAxzx544DHP1nKfvXHCN7PC\n6pnMS/PKp5tRmnnjjVeHLF68cAhAV9frQwBGjco/SFpvmpbwJU0AfgGsDQRwXkSc1ax4zKx4apHM\n83xp9MdTTz04+sgjt9oLQBIf/vAq/zJ27Ftev+SS528cdKyD3cEgLAaOiYh7Ja0CTJF0Q0Q83MSY\nzMz6pda/ADbc8B1v1PJmq3JNu/EqImZFxL3p/avAw8B6zYrHzKzTtcSdtpImAtsCdzY3EjOzztX0\nhJ/KOZcDR6eWvpmZ1UFTe+lIGg5cAfwyIn5bea1991/2fs/pcNyMRsRmZtY+TpsEN25aba1m9tIR\ncD4wLSK+3/ua113dqJjMzNrTcTOWbwxrv0prNbOksyvwCeA9ku5Jr32aGI+ZWUdrWgs/Im6jBa4h\nmJkVhROumVlBOOGbmRWEx9IxM2uiiy/+r4m33PJ/E2bPfmLMdtvt/ezJJ19d8yddlbiFb2bWi4UL\n0bPPMqo0/cILjHjtNYbW8hhrrTWha/Lk42bsvPMBM2u530qc8M2ssGbPZkR397LpWbMYWb78wQdZ\n7fTT2frJJxk9Zw4jTj+drf7+d9asZQz77vv5WXvv/a+zV1lljZoOu1yJE76ZFdYll7DxeeexSXc3\n3Hora556KtssWLCsBb/99szbbz+eOOUUdjjxRHZ65zuZtddezCktX7wYXXABG82bl5XHZ81i5EUX\n8dalAxrIOAZ/QlU44ZtZYX3hC8x44QVGf/Wr7HL55Wxy5JE8OGYM3eXrbLkl80rvd9iBl8qXDR1K\nDB/O0tNPZ8sZM1j5jDPYcrXVWDRkQJlVAzuJfnDCN7PCWmklunfZhVldXQxbfXW6Nt6Y18qXz5nD\niDPOYKu99+aJj32M6T/4AVs9+SSjS8uHDIFDD+XJtdfm9dNOY7tttuGFD32IAT69yi18M7O6ufVW\n1rzmGt765S9z39ChLC2Vd0rmzWP4u97Fc5Mn88xeezFnv/14Yu5clnvYyZw5jJw5kzHDhrH04YdZ\no1Te6T+38M3M6mbmTFY+6ige3GorXvna13hoyBCiq2tZDX/SJF474ACeK03vtRdzttuO+aXpxYvR\n97/PO3bdlefPOYfbN92Ul3/wA7boTw1/yZLFvPHGq0OWLl2ipUu71dX1+pAlSxbX6hSX4374ZlZY\nhxzC06X3K61E9xFH8Fh/th8+nPjyl3lo/Hi6ICvvPP88o/pTw//JT74y6dprz51Umv7wh1dZf7/9\njpx+1FHnPNqfWPJwwjczG4RSsoespl8+nceXvvTjGV/60o8bMuy7SzpmZgXhhG9mVhBO+Gatbwww\noWx6Qppn1i9O+Gatb3VgMrBBek1O88z6pdeLtpLOzrH9/Ij4zxrGY2Yrmkn27OdPpOlfpnmWU4SW\nLlrUpREjRtX/7qYGWbSoSxHq1yAOfbXwPwjcDUxJf/Z8TSFraZiZ1UpdylcvvLDm05dffs4ac+c+\nP7y7u7v6Bi2su7ubuXOfH3755ees8cILaz5dfYtlFFH5C0/SMRHxv31unGOdwZAUEEfUa/9mbWIC\nWePqijR9EHA5ndnKr9O5vj5k5MhzNp0w4eEthw1bOFpqwG2tdRJBLFky8o2ZM9/+wMKFX5wOK1Vo\n5evciFjhHHtN+K3ACd8MyFq4q7Ms6U0A5gELmhZRfW3A8uWrfrViDXpL+Hlr+MHyAz1ERHylhtGZ\nWe8WsHxyr0fLvpO+VDrpXGqqrxr+lPQaCWwHzAAeBbaB5QcPMrO21yo9gSaQlXF+mV4HsXxNP49W\nOZeWU7WkI+lOYLeIWJymhwO3RcROdQ/OJR2zRmqFUkqtWuetcC5NVLmkk6cf/urAqmXTpb8QM6sN\n31i1zAKWL1nNZPlk789qEPIk/O8BUyVdJOkiYCrw3fqGZVYorVCCqEUppRHyfFbtci4Nl6uXjqR1\ngR3T5J0RMauuUS07rks6VhTNLkG004XOap9VO51LnQywpCNpCPBeYOuI+B0wQtKOVTYzs8ZZF9ih\nbHqHNK8d1aJk47JQL/KUdH4E7AJ8PE2/muaZWW0MtgQxHjiU7Ff4jun9+H7G0AplpTxxuBfPIOTp\npXNPRGxb+jPNuy8itq57cC7pWDHUogSxI/DJ9P5i4K4BxNHsslKeONyLJ5eB99JZJOnNZzxKGgf0\na8AeM+tTtRJENWOANcum16Q5JYo8pZLBllMG+1kVWp6EfzZwFbC2pO8At+NeOmat5F1kgx1em14f\nTPP6o1GlkkaUbKopbC+evL103g7slSb/EhEP1zWqZcd1ScesunWBrcnuiIes6/R9wPP92EcjSyWN\nKNn0pQC9eAbeS+cCYFRE/DC9HpZ0cj1CNLMBeR6YVjY9jf4le2idUkkj4miVc224PCWdvYGLJH26\nbN4BdYrHrFZq0fWuETXpPKp1u8xTomjE55EnjsKWU1pBnoQ/B9gdOFjSj9JYOmatrhZd72pRk66F\nat0u55GNGf90el2e5tU6zmr7yBNHnnWsTvrTLVPAyWS1/PUiYqO6B+cavg1OLbreDbYmXSut0u2y\nw7szdoqBd8u8GrIB8CPiJOBU4MnaBmfWFIW947KCRn0W1Y7jv5M6qprwI+K/e0xfHRF71i8ks5rI\nUyuuRRfBRtSkdyC70/3i9Po4y9f086gWZ6MGJat2nMLeBdsIfT3T9vaI2FXSq2RPvCoXEbFqpe36\ndfCsB9AHgDkRsWWF5S7p2EDl7Xo32C6Cjejity5Zzf7uNL0D8Cy173bZqEHJqh3HZaNB6+cjDiNi\n1/TnKnWM6udkN3b9oo7HsGKqxWMB8+yjEY8ffJ7lk/vdva1o1pdeSzqS1ujrVYuDR8StwMu12JfZ\nABSpi2Atyle1KLdUO06R/k4arq+SzpMse3j5BixLzGOBpyLirTUJQJoIXO2SjjVBAe64XE4t7nAd\nbLml2nGK9ndSJ/0v6UwEkPRT4KqIuDZN7wscWKcoK9h3/2Xv95wOx81o3LGtw1Urx+RJPo2qr9db\nI0pTeY7TqDg6zGmT4MZNq62Vpx/+gxHxjmrzBsotfGthE8jKFlek6YPIbhQqT0I7kN0IdWma/jhw\nCf2rs+c5zmDV4hiNiNNqop8t/DLPSfpPsp9vAg4ha8GYdbqZZMmtvITRM7ndTXYtrPymqP5eVM1z\nnMEq3eFa2u9A7nCtxT6sifLcePUxYG2yIZKvTO8/3ucWOUm6FLgDmCRppqTP1GK/ZjUyBnhL2fRb\nqDyWTl9j0bfKjUS1GDCssIOOdYo+W/iShgFnR8Sh9Th4RNTki8OsTjYl+0VbKtccworlmtJY9L9P\n0x8ke0DQ9Wm61LOlZxmkPFGW90wpX8elEqupPhN+RCyRtKGkkRGxsFFBmbWI6cCvgPel6V+leeXu\nJyt1lsai/12aV5KnXONSiTVEnhr+E8Btkn4PvJ7mRUScWb+wzFrCAmB22fRsVixhPA8MZ1nCH+hY\n9O6ZYnWXp4b/OHBNWncVsvqjBzOyTlCLcearrbMZWSmotPyQNK8/POCY1UTVFn5EnNyAOMyaoTTO\nfKnhU+pSWWqh5ym1VFunUr/n6s8VXV616wB5rhOY5eqHvzbw78DmwOg0OxoxYqb74VsD1GKc+Woa\nMQ69BxyzMgMfD/8S4BFgI7IHoDyJB2+y+muFMkWtHgtYrWunWUPkSfhrRsTPgEUR8deI+Azg8fCt\n3hoxLnq1ceZrEUOpa+ef0+uQNK8/POCY1USeXjqL0p+zJO0HPEc2gJpZPTXi7tNnWb5f/VKWv4u8\nFjHk6dpZTbXrBO7WabnkSfinSFodOJZs7PpVgWPqGpVZY7zKit0uX63xMfJ07cyzDw84ZoOWp5fO\n1entPGCPukZjtkwj7j6t1rulFjH4LlprGX2Nh3922WRpXPzSeyLiK/UNzb10Cq5RQwbXYoz4vrTC\n0MdWOP3vpTMlvUaS3UU4A3gU2BYYUY8Qzco0YqCuaj1oPOCYdZS+HoByIYCkI4HdImJxmv4xcFtD\nojOrrzyDo5l1jDwXbVcnu1D7Upou/UQ1a3e16EFj1jbyJPzvAVMl3Zym3012A5ZZu6vWg8b1d+so\neXrp/FzSH4GdyC7Y/kdEzKp7ZGb1V60HjceosY5SdSwdAEnjgYlkXxClXjq31DUy3EvH6i5PC95j\n1FgbGuAzbSWdCnyUbJzv7rJFdU/4ZnXmG5asUPLU8A8ENvUTr6wfOqX2neemqU45V2tvx+dZKU/C\nf5ys370TvuXVKbXvPGPUdMq5Wms7fgzzK46yuhrzVgf4Gme++W/zq73sJM94+FcCWwN/YVnSD99p\na1UUqfZdpHO1OhrD/FN6zhtF16iRdI2aycS+So7fK58QDKyGD/w+vcr194k9ZmaFN47ZZ/S2bCRd\no9bixe572OG5HosWALcDtw72+Hm6ZV442INY4RRpwLAinatVd/h6PLNNbwuH0D10JhOn9LH9BXWI\n6U15SjqTgO+w4iMON6pnYOnYLum0pyJdyCzSuRqwHs+c1duyIXQP/Rpnzjuas/7YyyqDbqXnMZiS\nzs+Bk4AzyYZH/gwwtKbRWTtxglueu3Z2mDHMP6V0IbSSVHb5TR+7aEhSH4g8CX90RPxZkiLiKeBk\nSVOB/6pzbNaa8vRKcc8Va3nr89Q5vS1LZZe6lleaIU/C75I0FHhM0pfIHnG4cn3DshaW57F/jXg8\noVmfxjH7jJF0jept+U7c1XU5Hzm2kTE1W56EfzSwEvAV4FtkI2d+up5BmZmV2b2X+Zusz1Pb97Xh\nTCZ+sQ7xtK08Cf+tEfEPsp/jhwFI+gjw9zrGZa0rT68U91yx/ji8twWlhL4WL3ZXWn4PO8ykRx90\n612eXjr3RMS21ebVg3vptKQ8F219YdcqWSGxj2H+xqsxb/WduKurt42KVnaphX730pG0L/AvwHhJ\nZ7HsmbZjgMV1idLaQZ5eKe65Uky9jueyPk9NWIsXuzfmnyvkjsv5yGW0cM+WTtJXSec5smfaHpD+\nLCX8V4Bj6hyXmbWew8cwf+NKC8awYEypD3ov2847mrNOrGNslkNfz7S9D7hP0iVlz7NdA1g/Il5u\nVIBm1jiVxnIpWY15q6ebiiqV5hbgWnrLy3PR9gZJH0zrTgFekHR7RLiVb9aejh/H7HE9Z5a6MPYx\nSNfjdGDf9CLJ9RDziHhF0r8Cv4iIkyQ9UO/ArKP5om6dVbv9PyX1ORUWO6F3sDwJf6ikdYGPAP+Z\n5nm0TBsM34k7SOOYfcZwFg+vtGwI3UPb+fZ/q588Cf+bwPXA7RFxl6SNgUfrG5Z1ON+JW93hOW4q\nuqyPxU7otoI8wyP/BvhN2fTjZC2yQZO0D/B9ssHYfhYRp9Ziv2btoK+xXCB7gpF7tlgt9dUP//MR\ncV5fG+dZp49thwI/BN4LPAv8Q9LvI+LhgezP2kqR7sQ9fn2emlBpQSq71P3JcWYlfbXwj5f0Isv6\n35eLNP+rwIASPrAj8FhEPAkg6TKyPv9O+J0vz7NiW01v47mwPk99rLdlKal35MiL1n76Svi3APtX\n2f5Pgzj2eJZv0T0D7DSI/Vn7aMU7cXcHNqm0YByztyg9fq7S8o35Z+FGXbT21NeNV4fV+dg5e/rs\nW/als+d0OG5GfcKxAuhzkK4q47l41EVrWafBpBth02rr5emlUy/PktVySyaQtfJ7uO7qBsVjneFw\nYO2eM0t19EpJfXdu6zqaszzqorWt42DGcfBmY1iwX6X1mpnw7wY2kTSRbNyejwIfb2I81iby3P5f\nYZF7vFjhNS3hR8SS9ASt68m6ZZ7vHjoG1RN6uhD6XC+r+PZ/s15UTfiSRpF1m5tYtn5ExDcHe/CI\nuA64brD7sbZz+Dhmb1FpwXAWD0+3/k/pY3sndLMByNPC/x1Zl7kpQK8XtczKVRvLJd1U9Hgvqzih\nm9VBnoQ/PiL2rnsk1o6OX49n1htC99BKC33rv1lryZPw75C0VUTcX/dorOVUu/0/jbro3i1mbaCv\noRVKQyAPBT4j6QlgYZoXEbFVvYOz+hvH7DNK46BX4tv/zTpHXy380g1PpWEUynl45NZU6fb/TXKO\nuugSi1mH6+tO2ycBJF0cEZ8sXybpYuCTlbazuqo4nktpLJfebv0/jF+4D7qZ5arhv6N8QtIwoM8W\now1Kb7f/r70+T03oJam77GJmVfVVwz8B+DowWlL5QFeLGfgImdbLrf+Q3f6/Fi92b8w/F1dafjkf\n8aiLZjZgiui7HC/pexFxfIPi6XnsgDiiGccehN2BXSstGMfscSPpGtXLrf8AuPRiZoMlODciVhja\nPk9J5zeStusxbz7wVEQsqUl0bWaAY7kAdB3NWR5G18yaIk/CP4esZl/qh78l8BCwmqQjI+L6egXX\nZBVv/y91YUz9zytZgPulm1kLypPwnwM+GxEPAUjaHPgW8O/AlWSDn7Wlccw+o7dlI+kalZL6nAqL\nXUc3s7aTJ+FvWkr2ABExTdJmEfF4VmNvXWOYf8oYFoyptGwI3UPTTUX39rL5o7hvupl1kDwJ/yFJ\nPwYuI7sB6yPANEkjyXrsNNPhOW8q6o0TupkVRp6EfxhwFNkDywFuB75Gluz3rE9Yy1uPZ87qbYCu\nNOqie7aYmVVRtVtmM0mK9XlyCcBMJvqZomZmOQy4W6ak3YCTWPEBKBvVNMJeONGbmdVGnpLO+WTl\nnKlAxbFazMys9eVJ+PPSowjNzKyN5Un4N0k6jazPfWk8fCJiat2iMjOzmsuT8HcmG/9+hx7z31P7\ncMzMrF6qJvyI2KMBcZiZWZ0NqbaCpHUknS/pj2l6c0mfrX9oZmZWS1UTPnAh8CdgvTT9KHBMvQIy\nM7P6yFPDXysifi3peICIWCypkMMiF8WmcEI3vDkG0VBYMB2+027HMLPl5Un4r0paszQhaWey8fCt\nQ3XDmMeyYZ4BeFtZYm6nY5jZ8vIk/GOBq4GNJN0BjAMm1zUqMzOruTy9dKZIejewaZo1PSKaPUqm\nmZn1U18PMT+IrP+9yv4EmCSJiLiyAfFZEwyFBW/rUV9vx2OY2fJ6HS1T0oVkib6iiPhMnWIqjyEC\n2u0h5mZmTdXv0TIj4rC6RmSF1oheOuPg7GFl/8aXwJIX4Mu1PAa4x5G1jzwXbc1qrhG9dIbBsOfh\nzS7E69bp37t7HFm7yHPjlZmZdQC38AsmT/mhFqWQsfCTkcsu9LMQ4mX4Qml6Hqy6LaxaNm1mdZa3\nl05P4V467SlP+aEWpZCRoFllF/3X6fHvaDjwy7J57+2jg8BALYEl6/b44qr1McA9jqx99PUfeX/6\n/k/ohG8DtjK8skVZYly5DnXvelygrcQXaK1duJeOmVlB5PqpLmk/YHNgVGleRHxzoAeVdDBwMrAZ\n8E4/Pat2qtXo85QfqpVCqtXn07zlyjgLWd5LsMY6sEZvy6H6tYRq5+pumWbLq5rwJZ0LjAb2BH4K\nHAzcOcjjPgAcCJw7yP1YD9Vq9HkSUbWkWK0+D7AmzO0rjpFAtX1Uu5ZQ7VzdLdNseXm6Zb4rIj4F\nzI2Ib5A98nDTKtv0KSIeiYgZg9mHmZn1T54Wzxvpz9cljQdeAtapX0jFVYvSwEswdnxZqeT1AfR+\nyVMKWVq599ab5sLYrcvimFshju4q+1gKQ6ZlHXpyHXMgXI6xIsmT8P8gaSxwGjAlzftptY0k3UDl\nL4YTIuLqvAHum/UWAmBPmH4cdOwvg1qUBkaCHiubfusAkmS1UshClj3+rDTd03DQpWVJ/j094lgI\njK+yj8VviJd2AAAMDElEQVQQHy/bbnGPL41q1yPydMusxWfubpnWbKfBpBtzVF7yJPz/iYgu4ApJ\n15BduO2qtlFEvC/Hvqu6LhuL33ISLB1dlthUh7p1tfo8wBBYunlZHEN6xDEKllarr68BL983iOsR\n7pZpRXEczChvDAv2q7RenmRwB7AdQEr8XZKmlubVQM1/prerPOWYwZYgGlXCaEQ5ppo85+o7fq1I\n+rrTdl2yX+4rSdqOZePirwqsNJiDSjoQOAtYC7hG0j0Rse9g9tkJ8pRjqpUgqpUx8pQwqu0jTwmj\nWjkmT7llsKWSPOfaiDt+zVpFXy389wOHkZVazyibvwA4YTAHjYirgKsGs49OVItyTC3KGNX2kecX\nQbVyTJ44G1EqacQdv2atoq87bS8CLpI0OSIub2BM1gFeg1UfgtXKpgvbcnZPIGsVeVqQt0k6Hxgf\nEftI2hzYJSLOr3NshdOIMkejepQsBj5RluSb8RDkPOfaiM/DN2ZZq8iT8C8Efg6cmKYfBf4PcMKv\nsUaUORrVslwdXrmnyUkuz7m6pW1FkifhrxURv5Z0PEBELJZUl2FmO51/2ptZM+VJ+K9KWrM0IWln\nYH79QupcRfpp75uRlvFnYa0iT8I/luzmp40k3QGMAybXNSpre/7lsow/C2sVVRN+REyR9P/IbtsV\nMD0imnENzszMBiHP8MijgaOA3ch6Xdwq6cfprlvrh0b8tPd1AjPrTZ6Szi+AV8jujBVwCHAx2bj4\n1g+NSLxFuk5gZv2TJ+FvERGbl03fKGlavQIyM7P6yPMAlKmSdilNpF46U/pY38zMWlCeFv4OwO2S\nZpLV8DcApkt6AIiI2KqeAVr/uAugmfUmT8Lfp+5RWM34Aq2Z9SZPt8wnGxCHmZnVWZ4avpmZdQAn\nfDOzgnDCNzMrCCd8M7OCcMI3MysIJ3wzs4JwwjczKwgnfDOzgnDCNzMrCCd8M7OCcMI3MysIJ3wz\ns4JwwjczKwgnfDOzgnDCNzMrCCd8M7OCcMI3MysIJ3wzs4JwwjczKwgnfDOzgnDCNzMrCCd8M7OC\ncMI3MyuIpiR8SadJeljSfZKulLRaM+IwMyuSZrXw/wRsERFbAzOArzcpDjOzwmhKwo+IGyJiaZq8\nE1i/GXGYmRVJK9TwDweubXYQZmadbli9dizpBmCdCotOiIir0zonAosi4le97Wdf2L/0fk+YflxW\nAjIzs+Q0mHQjbFptPUVEI+JZ8cDSYcDngL0ioquXdSLgiIYGZmbW5gTnRoR6zq9bC7/PYKR9gOOA\nd/eW7M3MrLaaVcM/G1gFuEHSPZJ+1KQ4zMwKoykt/IjYpBnHNTMrslbopWNmZg3ghG9mVhBO+GZm\nBeGEb2ZWEE74ZmYF4YRfY6fBpGbHUA+deF6deE7g82o3jTwvJ/way3N7czvqxPPqxHMCn1e7aeR5\nOeGbmRWEE76ZWUE0bfC0PCS1bnBmZi2s0uBpLZ3wzcysdlzSMTMrCCd8M7OCcMKvMUmnSXpY0n2S\nrpS0WrNjqgVJB0t6SFK3pO2aHc9gSdpH0iOSHpX0H82OpxYkXSBptqQHmh1LLUmaIOmm9O/vQUlf\naXZMtSBplKQ7Jd0raZqk79b7mE74tfcnYIuI2JrscYxfb3I8tfIAcCBwS7MDGSxJQ4EfAvsAmwMf\nl/T25kZVEz8nO6dOsxg4JiK2AHYGvtgJf1/p4U/viYhtgK2A90jarZ7HdMKvsYi4ISKWpsk7gfWb\nGU+tRMQjEdEpzxPeEXgsIp6MiMXAZcABTY5p0CLiVuDlZsdRaxExKyLuTe9fBR4G1mtuVLUREa+n\ntyOAocDceh7PCb++DgeubXYQtoLxwMyy6WfSPGtxkiYC25I1ptqepCGS7gVmAzdFxLR6Hq8pT7xq\nd5JuANapsOiEiLg6rXMisCgiftXQ4AYhz3l1CPdFbkOSVgEuB45OLf22l6oB26RrfddL2iMibq7X\n8ZzwByAi3tfXckmHAf8C7NWQgGqk2nl1kGeBCWXTE8ha+daiJA0HrgB+GRG/bXY8tRYR8yVdA+wA\n3Fyv47ikU2OS9gGOAw5IF2U60Qp38LWZu4FNJE2UNAL4KPD7JsdkvZAk4HxgWkR8v9nx1IqktSSt\nnt6PBt4H3FPPYzrh197ZwCrADZLukfSjZgdUC5IOlDSTrJfENZKua3ZMAxURS4AvAdcD04BfR8TD\nzY1q8CRdCtwBTJI0U9Jnmh1TjewKfIKsF8s96dUJvZHWBW5MNfw7gasj4i/1PKCHVjAzKwi38M3M\nCsIJ38ysIJzwzcwKwgnfzKwgnPDNzArCCd/MrCCc8K0lSNpD0grDN/Q2vwbHO6B8xEVJN0vaPkeM\n8yX9ocp6J9QqzrS/QQ0jIOlkScem99+QNOg7wEsxSdo4De+7YLD7tPpzwreiOpBsaOSSvDek3BIR\n+1VZp9ZDYue+WUZJb9tHxEk1urkn0v4eT8P7WhtwwrdcJK0s6ZrUmntA0kfS/O1T6/huSX+UtE6a\nf7Ok76e7Ih+Q9M40f0dJd0iaKul2SZP6GcMF6aERUyV9MM0/LD1s5jpJMySdWrbNZyVNT9ucJ+ls\nSbsA+wOnpf1slFY/OK03Pc+45JLWlXRL2TnuJul7wOg07+K03m/T5/OgpM+Vbf+qpG+nz/RvktZO\n89+apu+X9O2y9VeR9GdJU9Ky0vlPTDFfRPbcggmSTkzzbgU2JSVoSRdKOij9vZXuWn1A0tK0fOP0\nOd6dzm3TvmKyNhMRfvlV9QUcBJxXNr0qMJzsVv4107yPAuen9zcB56b3uwMPpPdjgKHp/XuBy9P7\nPchuLe953DfnA98BDk3vVwemAysBhwGPp32PBJ4kG+54PeCJtO4wsoe3nJW2/znw4bLj3ASclt7v\nC9zQVyxp+t/IRhKFrPG0Snq/oMd2Y9Ofo8kScml6KfCB9P5U4MT0/vfAJ9L7o0r7IxsvfUx6vxbw\naHo/EegGdkzT2wP3A6PSZ/Io8G+VzjvN+x/g1PT+L8Db0vudgL/0FVPZPhb0/Lz8ar2XR8u0vO4H\nTk8t2D9ExG2S3gFsAfw5VRGGAs+VbXMpZA/mkLSqpFWB1YBfSHobWatzeD9ieD+wv6SvpemRwAZp\nP3+JiAUAkqaRJcFxwF8jYl6a/xug/BdFz9LHlenPqWn7av4BXKBsJMffRsR9vax3tKQPpfcTgE2A\nu8iGz74mzZ9CNngWwLvISk4AvyT7MoDsS+W7knYn+7JYr/SrAHgqIu5K73cHroxs8L4uST0Hhnvz\nvCV9FNgOeJ+y4Yd3AX5TVhUaUSUmayNO+JZLRDwqaVvgA8C3Jf0FuAp4KCLe1Y9dfYssOR8oaUP6\nPxTshyPi0fIZknYCFpbN6ib7t92z9t1rbTsp7aO0fZ/SF9nuwH7AhZLOjIiLe8S2B9kw2TtHRJek\nm8ha3pA9uq9kaY5jHkrWst8uIrolPVG2r9d6nFf5uVYc3TR9YZ8E7B4RIWkIMC8itq0Sh7Up1/At\nF0nrAl0RcQlwOtlTh6YD4yTtnNYZLqn8QuhH0/zdyBLJK2SloNKvgP6O5ng98OYDrNMXEFROaEHW\nAn+3pNUlDSMrS5WS/IIUy4BJ2gB4ISJ+RjZ8bymexel4pGO8nJL9ZmSjjVZzO/Cx9P7QsvmrAnNS\nsn8PsGEv298CfEjZQ7LHkH0hlQtlw/JeCnwyIl4CSH8/T0ianM5PkraqEpO1ESd8y2tL4E5J9wD/\nDXw7sufBTgZOVTbE6z1kJYGSLklTgR8Bn03z/oesLDGVrARU3squ1BslyuZ/CxieLhw+CHyjwjrL\nNox4jqzufxdwG1k9f35afBlwXLoAulHPbXuJpac9gHvTuRwM/CDNPw+4P120/SMwLJWZvgv8rZdj\nlJ/D0WQP6r6f7DpEaf4lwA5p/ifJnu26wr4i4h7g18B9ZI/YvIsVfZCsHPazdOF2app/KPDZ9Pf5\nYFqvr5isjXh4ZKuLVLo4NiKmVl25vnGsHBGvpRb3lWQXlX83wH3tQXZO+9cyxk4gaUFEjGl2HNY3\nt/Ct052cfpU8APxzoMk+WQi8Q1VuvCoSSRulXwOzmh2LVecWvplZQbiFb2ZWEE74ZmYF4YRvZlYQ\nTvhmZgXhhG9mVhBO+GZmBfH/ARpKJfjkqLWeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108bc1748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEPCAYAAABMTw/iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHGW5/vHvnQQIYYvIqmwxARECAmLYgo6CApFV/AmK\nKxJUIAQPKqKeQ/SHiAdQAfEAUXABVASOoAYIImFRdgIkhF2WBAlhJwkEEvKcP6qG9Ey6Z3pmurqq\nq+/PdfWV7urqep8E5ul3nnoXRQRmZlZeg/IOwMzMsuVEb2ZWck70ZmYl50RvZlZyTvRmZiXnRG9m\nVnKZJnpJEyXNkDRT0sQs2zIzs+oyS/SSRgOHAe8H3gvsLWlkVu2ZmVl1WfboNwdujYhFEfEmcD3w\n8QzbMzOzKrJM9DOBXSWtKWkY8DFggwzbMzOzKoZkdeGIeEDSj4CpwEJgOrA0q/bMzKw6NWutG0kn\nAU9GxNkVx7zQjplZP0SE6j03sx49gKR1ImKepI2AA4Adup/Tl2DzImlSREzKO47eOM7GcpyN1Qpx\ntkKM0PdOcqaJHrhE0tuBxcAREfFKxu2ZmVk3mSb6iPhAltc3M7PeeWZsfablHUCdpuUdQJ2m5R1A\nnablHUCdpuUdQJ2m5R1AHablHUAWmnYztmrjUrRCjd7MrEj6mjvdozczKzknejOzkst61E3TSKPH\nwYZHw2pDYf4imH1GxMwpecdlZpa3UiT6JMnvdDpMHrXs6PiR0mic7M2s3ZWkdLPh0V2TPCSvN5qQ\nTzxmZsVRkkS/2tDqx1ddublxmJkVT0kS/fxF1Y8veK25cZiZFU9JEv3sM2D8I12PHfYoPHlmPvGY\nmRVHaSZMJTdkN5oA664La4+CKQf7RqyZlVFfc2dpEv2yazIEmAdsGcHTjby2mVkRtP3M2AiWAH8H\nds87FjOzIihdok9dA3w07yDMzIqgdKWb5LqMAG4G1o/Au1iZWam0fekGIILHgPnAVnnHYmaWt1Im\n+pTLN2ZmlDvRT8WJ3sysnDX65NqsAcwB1onAM2TNrDRco09F8DJwDzA271jMzPJU2kSfcp3ezNpe\n2RO96/Rm1vbKnuhvBzaSWC/vQMzM8pJpopd0vKT7JM2QdJGklbJsr7t0OYTr8HIIZtbGMkv0kjYB\nxgPbRcRWwGDg4Kza64HLN2bW1rLs0b8CLAaGSRoCDAOeyrC9WqYCH5HIZBinmVnRZZboI+IF4DTg\nSeDfwEsR8bes2qsdB/8CFgKjm922mVkRDMnqwpJGAscAmwAvA3+UdEhEXNjtvEkVL6dFxLQMwukc\nZjkjg2ubmWVKUgfQ0e/PZzUzVtJBwEci4rD09WeBHSPiyIpzMpsZ2zUWDgC+EsEeWbdlZpa1Is2M\nfQDYUdLKkkQy8mVWhu315DpgZ4mhObVvZpabLGv09wC/Ae4A7k0Pn5tVez3HwktpDF4OwczaTmkX\nNVu+LSYBwyL4ZjPaMzPLSpFKN0Xj8fRm1pbaKdHfBmwssW7egZiZNVPbJPp0OYRpeDkEM2szbZPo\nUy7fmFnbacdE7+UQzKyttFWij+BR4DVgy7xjMTNrlrZK9CnvOmVmbaUdE73r9GbWVtpmwtSyNhlO\nsqLmOhEsambbZmaN4AlTvUiXQ7gP2CXvWMzMmqHtEn3K5RszaxtO9GZmJdd2NfqkXVYAngU2i2Be\ns9s3MxsI1+jrEMFivByCmbWJtkz0KZdvzKwttHui93IIZlZ67ZzoHwXeALbIOxAzsyy1baKPIHD5\nxszaQNsm+pQTvZmVXlsOr1zWPm8DngDWjuD1vOIwM+sLD6/sgwheBGbh5RDMrMTaOtGnXL4xs1Jz\noneiN7OSyzTRS3q3pOkVj5clHZ1lm/1wK/AuibXzDsTMLAtNuxkraRDwFDAmImanx3K9GdtJ4nLg\n9xH8Lu9YzMx6U+SbsbsDj3Ym+YJx+cbMSquZif5g4KImttcXXg7BzEprSDMakbQisA9wXJX3JlW8\nnBYR05oRUzePAEuA95AMtzQzKwxJHUBHvz/fjBq9pP2Ar0bEnt2OF6JGDyBxLjArgp/mHYuZWU+K\nWqP/FBT+Rqfr9GZWSj326CUJ2GAgN1AlrUKyzMCIiJjf7b0i9ejXBB7HyyGYWcFl0aO/cgDxEBEL\nI2Kt7km+aCJ4Abgf2DnvWMzMGqnHRB9Jd/9OSWOaFE/eXL4xs9Lp9WaspAeBUSTll4Xp4YiIrQfc\neIFKNwASuwI/jeB9ecdiZlZLX3NnPYl+k/Rp54kCiIjH+x7ectcuWqJfAXgOGBXBs3nHY2ZWTcNr\n9GlCHw7sSzIWfo1GJPkiimAxcD2wW96xmJk1Sq+JXtJE4AJgbWBd4IICLkzWSK7Tm1mp1FO6mQHs\nGBEL09erALdExFYDbrxgpRsAic2Aa4GN0n1lzcwKJasJU0trPC+jh0n+jpvnHYiZWSPUs9bN+cCt\nki4juRG7P3BeplHlKIKQ3irf3J93PGZmA9XbzNhBwE7AImAsycibGyNiekMaL2DpBkDi/wFfiOBj\necdiZtZdFsMr746IbQYcWfVrFzXRvx14DC+HYGYFlEWN/m+SPpGue9MWIngeeIDktxkzs5ZWT49+\nATAMeJOkhAPJzNjVB9x4QXv0ABInAoMi+HbesZiZVWpojz6t0e8REYMiYoWIWC19DDjJtwCPpzez\nUnCNvgaJFYFngZERPJd3PGZmnVyjb5AI3gBuwMshmFmLc42+BxITgG0i+FLesZiZdWr48MostUCi\nfzdwDbCxl0Mws6Loa+7sdWZsekP2EJKtAL8vaSNgvYi4bQBxtojRI+Fja8PTt0rPvgCzz4iYOSXv\nqMzM+qKe0s3ZJGu/fDgiNpe0JjA1IrYfcOMF7tFLo8fBTqfD5FHLjo5/BG6e6GRvZnnK4mbsDhFx\nBPAaQES8AKzQz/hayIZHd03ykLzeaEI+8ZiZ9U89if4NSYM7X0ham/KvYAmsNrT68VVXbm4cZmYD\nU0+iPxP4X2AdSScB/wB+mGlUhTB/UfXjC15rbhxmZgPT683YiLhA0p0sG0++X0S0wfK9s8+A8SO7\nlm+OmgtPnplfTGZmfden4ZWSvhwR5/Th/OHAL4AtSZY4PjQibql4v7A3Y6HzhuxGE5JyzQrD4Asb\nwEdGeEVLM8tTpuPoJU2PiG37cP6vgesj4jxJQ4BVIuLl/gabN4krgJsi+O+8YzGz9pV1oq973RtJ\nawDTI+JdPZzTaol+U+BmYHQEc/OOx8zaU1Z7xnbauw/njgCelXS+pLskTZY0rI/tFUoED5Nso3hS\n3rGYmdWrZo9e0rEVL4Nkv9jO50TEj3u8sLQ9Se9354i4XdJPgVci4r8qzgngexUfmxYR0/r6l2gm\nidWBB4F9Irgj73jMrPwkdQAdFYdOaEjpRtIkkqT+buD9wBUkyX5v4LaI+Ewvga0H3BwRI9LXY4Fv\nRcTeFee0VOmmk8SXgEOBsV4Dx8yarWGlm4iYFBHfAzYEtouIYyPiP4D3ARv3duGImAvMlrRZemh3\n4L56Ayu4XwFDgYNzjsPMrFe9jqMH1gEWV7xenB6rxwTgQkkrAo8CX+xbeMUUwZsSxwAXSlwewat5\nx2RmVks9i5p9BzgIuIykdLM/8IeIGPANyVYt3XSS+D3wQAST8o7FzNpHJsMrJb0PGJu+vCEipvcz\nvu7XbfVEvxFwF7BdBE/mHY+ZtYeshlcOA+ZHxOnAHEkj+hVdyaTJ/SzgR3nHYmZWSz2lm0kkN2Df\nHRGbSXoncHFE7DLgxlu8Rw8gsQpwP/DpCG7KOx4zK78sevQHAPsBCwEi4ilgtf6FVz4RLASOA06X\n+jwBzcwsc/Ukptcj4q315yWtkmE8rer3JBunfyHnOMzMllNPov+jpHOA4ZIOB64lWZHSUumkqYnA\nD9KZs2ZmhdFjjV6SSCZMbQ58ND18dURc05DGS1CjryRxHvBsBMflHYuZlVdDh1emiX5GRIxuRHBV\nrl+2RL8eMBPYKV0Azcys4Rp6MzaSb4E7JY0ZcGRtIF26+BTg1LxjMTPrVM/wygeBUcATpCNvSL4D\nth5w4yXr0QNIrESyps9XI2hIicvMrFLDZ8ZK2qTa8Yh4vC+B1bh26RI9gMT+wInANhEsyTseMyuX\nho+jj4jH06T+KrC04mG1XQ7MBb6cdyBmZvX06PcFTgPeAcwjWaL4/ojYcsCNl7RHDyCxFclQ1PdE\n8Hze8ZhZeWQxM/ZEYCfgoXQTkd2AW/sZX9uIYAbwR/DKlmaWr3oS/eKIeA4YJGlwRFwHbJ9xXGXx\nX8BBEgP+7cfMrL/q2XjkRUmrATeSbCIyD1iQbVjlEMHzEicCP5HYw9sOmlke6qnRrwq8RtL7PwRY\nHbgwIgZcdy5zjb6TxArAPcC3Irgi73jMrPVlsvFIVtoh0QNI7AH8DBgdwet5x2Nmra3hN2MlLZA0\nP328LmmppFcGFmZ7ieBq4AHg6LxjMbP206cevaRBwL7AjhHxrQE33iY9egCJzYB/AltG8Eze8ZhZ\n62pK6UbS3RGxTZ8/uPx12ibRA0icArwtgsPyjsXMWldfc2evo24kHVjxchDJtoKv9SM2gxPh2n9J\n52wBvAHzF8HsMyJmTsk7MDMrr3qGV+4Dbw0LXAI8TrK1oPXZ6F1gtyVw8U7Ljo0fKY3Gyd7MspL5\nqBtJjwOvAG+STL4aU/Fem5Vu9roKrtxj+XfGXRUxZa/mR2RmrSiL0s2ZJD36zot2eR4RvY0kCaAj\nIl6oN6jyWm1o9eOrrtzcOMysndSzBMJQYDvgIeBhYFtgReAO4M4622mbXnvP5i+qfnyB73mYWWbq\nqdFvDYyNiMUAkv4HuCki6l2CN4C/SXoTOCciJvcv1DKYfQaMHwmTRy079vWFMOdn+cVkZmVXT6If\nTrLsQeeSB6ulx+q1S0Q8LWlt4BpJD0TEjZ1vSppUce60iJjWh2u3lIiZU6TRwLgJSbnm1dfhiA3h\n1PcDf807PjMrJkkdQEe/P1/HWjdfJFlq9zqSEswHgUkR8as+NyadACyIiNPS1211M7YaiXWB24Bv\nRHBx3vGYWfFlMmFK0vrADiRlmFsjYm6dwQwDBkfEfEmrAFOB70XE1P4EW1YS2wDXAHtG1H3fw8za\nVBZr3ewCzI+IP5GUcL4paeM6r78ucKOku0k2K/lLZ5K3ZSK4m2TbwT9JrJ93PGZWLvWUbmYA7wW2\nAn4F/AL4ZER8cMCNu0ffhcR3SSaodUR49rGZVZfFVoJLImIpsD9wVkScRXJD1hrvB8BjwGTJQ1LN\nrDHqSfTzJX0b+AzwF0mDgRWyDas9pTtQHQpsDhyXczhmVhL1JPqDgEXAoelN2HcCp2QaVRuL4FWS\ntYSOkrymkJkNXF/Xo987Iv7SsMZdo69JYgzJ2PoPRzAj73jMrDiyqNFX+v99PN/6KYLbgInAFRJr\n5x2PmbWuviZ6a6IILgIuAi6VWDHveMysNfW1dDMmIm5rWOMu3fRKYhBwGfAcMD69YWtmbazhM2Ml\nDQE+BmzCsrVxIiJ+3N8gK67tRF8HiVWBfwDnRXB63vGYWb4avh498GeSrQNnAEv7G5j1XwQLJPYF\nbpF4IIKr847JzFpHPT36eyNi60wad4++TyTGkpRxPhDBA3nHY2b5yGLUzVRJVba/s2aL4CbgeJKR\nOGvmHY+ZtYZ6evQfBy4g+VJYnB6OiFh9wI27R98vEj8m2RBmr4i3/puYWZvI4mbs48C+wMx0zZuG\ncaLvH4khJPdOHolgQt7xmFlzZVG6eRK4r9FJ3vovgiXAwcDuEvVu6WhmbaqeUTePAddJuhJ4Iz3W\nkOGV1n8RvJyOxLlJ4sEIpuUdk5kVU72J/jFgxfQh8KSdIojgYYlPw7WXSWfNhCFLYf4imH1GxMwp\necdnZsXQa6KPiElNiMP6bfRKsNsSuGzXZcfGj5RG42RvZlDfzdjrqhyOiPjwgBv3zdgBk/a6Cq6s\nMvx13FURU/ZqfkRmlrUsZsZ+o+L5UOBAYElfA7OsrDa0+vFVV25uHGZWVPWUbu7odugmSbdnFI/1\n2fxF1Y8P8mqXZgbUMbxS0poVj7Uk7QkMeLKUNcrsM2D8I12PHfk0HL6FxCfyicnMiqSe0s1dLBtl\nswR4HPhSVgFZ30TMnCKNBsZNSMo1C16DJ8+Es54mWSphJPDfXt7YrH3VvBkraQwwOyKeTl9/gaQ+\n/zhwQkS8MODGfTM2UxIbkMygvRP4qpdLMCuHRs6MPQd4Pb3oB4AfAr8CXgbO7UNAgyVNl/Tnej9j\njRHBHGBXYF3gSonhOYdkZjnoKdEPqui1HwScExGXRsR3gU370MZEYBaeZJWLCBYA+wP3Af+UGJFz\nSGbWZD0l+sGSVkif7w5Ujqevp7aPpA2AccAvSGbUWg4ieDOCicDPSZL9TnnHZGbN01Oi/x1wvaQr\ngFeBGwEkbQq8VOf1f0IyDt8LohVABD8juZF+hcRBecdjZs1Rs2ceET+Q9HdgPWBqxeqVgt6XxpW0\nNzAvIqZL6ujhvEkVL6dFxLQ64rZ+imCKxO7An9MROT/0iByzYktzaEe/P9/bEgj9vrB0EvBZkiGZ\nQ0nG3l8aEZ+rOMejbnIi8Q6SETn3Al+OeGtlUjMruIZvPNIIkj4IfD0i9ul23Ik+RxKrABcCawAH\nRjDgIbNmlr0sNh5pFJcHCiaChSRzI+4iuUk7KueQzCwDTenR12zcPfrCkPgKMAn4RLoJuZkVVJF7\n9FZgEZwNfB64LNnMxMzKwj1660JiK+DPcP6NcPHayTLI3rXKrEiyWI/e2kgEM6QDvwOjfglXrrTs\nHe9aZdaqXLqxKl79LPxopa7HJo+CjXqdP2FmxeNEb1XU2rVquPchMGtBTvRWRa1dqzbZXuIIicHN\njcfMBsKJ3qqotmvVYY/CS8eQrGR6uxdGM2sdHnVjVUmjxyU1+WW7ViW7WSHgU8ApwFTguAjm5Rut\nWXsp5BIINRt3om9ZEqsD/0Uy9v77wP9EsCTfqMzagxO9NZXEFsDPgLcDR3pWrVn2nOit6dJyzieB\nU0k2qPlmBHPzjcqsvLwEgjVdBBHBH4AtgLnATIljJE/IMysC9+it4SQ2B84k2bTmyAhuyDkks1Jx\n6cYKIS3nHAj8mGQbym/A6G1gw6O9fo7ZwHitGyuEdHvCSySuBL4Lf78fdn8DfrrWsrO8fo5ZM7hH\nb00hHXgjXDp2+XfGXRUxZa/mR2TWunwz1gpq8JvVj3v9HLOsOdFbk9RcP2eMxAUSO6d1fTNrMCd6\na5Ja6+fc/RmSPWt/DUyXOFxi1RwCNCst1+itaWqtn5O8xyBgN+AI4APAhSTLKtyfY8hmheThldby\nJDYEDgcOA+4Hfg5cHsHiXAMzKwgneisNiRWBA0h6+aOAycDkCJ7KNTCznDnRWylJjAa+AnyaZD2d\nnwN/h9F7eRKWtZtCJXpJQ4HrgZWAFYHLI+L4ived6K1PJFYDDgGOhKnD4a9D4fTKSViPwM0Tneyt\nzAqV6AEkDYuIVyUNAW4Cvh4RN6XvOdFbvyRDMQ++BX4/Zvl3PQnLyq1wE6Yi4tX06YrAYOCFrNu0\n8kuWWFj6WvV33ztW4kSJXbyCplkTEr2kQZLuBp4BrouIWVm3ae2i1iSsx+8j6VScBcyTuFjiixLr\nNzE4s8LIvLcTEUuBbSStAVwtqSMipnW+L2lSxenTKt8z69nsM2D8SJg8atmxwx6FGd+PYApwvMQ7\ngD2AvYDTJJ4Arkwft3jIprUCSR1AR78/38xRN5L+E3gtIk5NX7tGbwPS0ySs5c9lCLADSdLfC3gX\ncC1J0r+qc9hmck2P5LHiKtTNWElrAUsi4iVJKwNXA9+LiGvT953oLTcS65H09vcEPgrMgd8+BLfs\nBGe9c9mZHsljxVK0RL8VyRomg9LHbyPilIr3neitECQGA2Pgy7+Fc0Yuf8aBN8ClHek6+2a5KtTG\nIxExA9guyzbMGiGCN4GbpRfnAFUS/Xt2AF6UmA5MJ1mI7S7gwfSzZoXloWdmXdQayXPXdcBngW1J\nOi/7ACcA60vcy7LEfxcwK4I3Kj/tur/lyYnerItaI3mePDOC54Br0gcAEmsA25Ak/w8BxwIjJGbx\nVuI/dijsfBScW/GbgrdRtObxWjdm3fRlJE/1z7MKsDVJ8t8Ovv4JOLXKTlqH3A4XHgw8GcGSRsVv\n5VeoGr1ZK0qTer972hEsBG5OH0hPjgQ+uPyZG2xKskDbehJzgH9Ve0TwYrV2XA6yejnRm2WuVt1/\nxi0R7CWxErARybj+kemfO3S+llgCPEqXL4AT1oKxh8PZGy+73sDLQf7yKCcnerPM1a77A0TwOvBw\n+ugi3Uf37Sz3JTB/fzh7za5nTx4FE86VuIBkyZFngLkVz1+IYGmtKJMkv9PpXeP0vYQycKI3y1jE\nzCnSaGBcn+v+6bj959LHbZ3HpTk1ykFLFgCvAJsAOwLrVjxWk5hH9S+BZ2CrY7omeUhej5vAAEpZ\n/i0hf070Zk0w0Lr/8mqVg554LIKTqr2TlojWIUn667HsC2AEsCNsvFn1a26zq8QNwEsVj5e7ve7+\neDmCJVn9ltDoL4+yfxk50Zu1pJ7LQdWkJaLZ6WM50j1XkSwJ0c0jdwP/CQwH1kj/HA68E9iy4nXl\ne2tIvAZ7D4GTh3a93uRRcNTZEn8EFgKvpn9WPmodexVG79nIL49W+TIaCCd6sxY0kHJQbbW+PGad\nFMH1fblSem9hVZg9laSE1M3SRcC/gVWAtwEbpM9XAYZVPO/+GAr7Bfyg2xLrk0fBsRdI3Am8nj4W\ndfuzxvP3HV29ZHXwdyXmAovreCypXB6jaPc7nOjNWlSjy0GN/PJIk9586YWXq5/x+KMRnNbX60oM\ngkenAbsu/+6LTwCnkGxduhIwtMbzNboeW2u96q2NGA38ElihjsfgdHRUmvj3WRl+uELX6w38fkd/\nOdGb2Vsafy+h7yWmnkSwVHr51ervzp0bwdS+XlOaVaNkdc8/IqhrS8rkC4ghvJX4n/grsPPyZ666\ncl/jawQnejPLTHNLTP378mjE9dJhq2+kD6QX51c/c0GN7S+z5SUQzKzlDHSZiuZcr3uN/rBH4Zaj\nG1GjL9R69L027kRvZiXV6C+Prtd2ojczK7W+5s5BvZ9iZmatzInezKzknOjNzErOid7MrOSc6M3M\nSs6J3sys5DJN9JI2lHSdpPskzZR0dJbtmZnZ8rLu0S8GvhYRW5KsYHekpPdk3GbDSerIO4Z6OM7G\ncpyN1QpxtkKM/ZFpoo+IuRFxd/p8AXA/8I4s28xIR94B1Kkj7wDq1JF3AHXqyDuAOnXkHUCdOvIO\noA4deQeQhabV6CVtAmwL3NqsNs3MrEmJXtKqwCXAxLRnb2ZmTZL5WjeSVgD+AlwZET/t9l5+C+2Y\nmbWwwixqJknAr4HnI+JrmTVkZmY1ZZ3oxwI3APfCW/spHh8RV2XWqJmZdZHrMsVmZpa93GbGStpT\n0gOSHpZ0XF5x9KTVJnxJGixpuqQ/5x1LLZKGS7pE0v2SZknaMe+YupN0fPrffIakiyStlHdMAJLO\nk/SMpBkVx9aUdI2khyRNlTQ8zxjTmKrFeUr63/weSZdJWiPPGNOYlouz4r1jJS2VtGYesXWLpWqc\nkiak/6YzJf2op2vkkuglDQZ+BuwJbAF8qqATqVptwtdEYBbLymRFdDowJSLeA2xNMreiMNJhwOOB\n7SJiK2AwcHCeMVU4n+RnptK3gGsiYjPg2vR13qrFORXYMiLeCzwEHN/0qJZXLU4kbQh8BHii6RFV\nt1yckj4E7AtsHRGjgVN7ukBePfoxwCMR8XhELAZ+D+yXUyw1tdKEL0kbAOOAXwCF3LUr7cXtGhHn\nAUTEkoh4OeewunuF5At+mKQhwDDgqXxDSkTEjcCL3Q7vSzLggfTP/ZsaVBXV4oyIayJiafryVmCD\npgfWTY1/T4AfA99scjg11Yjzq8AP0/xJRDzb0zXySvTvBGZXvJ6THiusFpjw9RPgG8DS3k7M0Qjg\nWUnnS7pL0mRJw/IOqlJEvACcBjwJ/Bt4KSL+lm9UPVo3Ip5Jnz8DrJtnMHU6FGjI3qmNJmk/YE5E\n3Jt3LL3YFPiApFskTZO0fU8n55Xoi1xaWE7RJ3xJ2huYFxHTKWhvPjUE2A74eURsByykGKWGt0ga\nCRwDbELy29uqkg7JNag6RTKyotA/W5K+A7wRERflHUt3aafj28AJlYdzCqc3Q4C3RcSOJB28i3s6\nOa9E/xSwYcXrDUl69YWTTvi6FLggIv6Udzw17AzsK+kx4HfAhyX9JueYqplD0lu6PX19CUniL5Lt\ngX9GxPMRsQS4jOTft6iekbQegKT1gXk5x1OTpC+QlBeL+sU5kuQL/p70Z2kD4E5J6+QaVXVzSP7f\nJP15Wirp7bVOzivR3wFsKmkTSSsCBwFX5BRLTemEr18Cs7rP6i2SiPh2RGwYESNIbhz+PSI+l3dc\n3UXEXGC2pM3SQ7sD9+UYUjUPADtKWjn97787yQ3uoroC+Hz6/PNAITsjkvYk6XnuFxGL8o6nmoiY\nERHrRsSI9GdpDslN+SJ+ef4J+DBA+vO0YkQ8X+vkXBJ92lM6Cria5IfoDxFRqNEXqV2AzwAfSoct\nTk//hy26Iv/6PgG4UNI9JKNuTso5ni4i4h7gNySdkc467bn5RbSMpN8B/wTeLWm2pC8CJwMfkfQQ\nyQ/+yXnGCFXjPBQ4E1gVuCb9Ofp5rkHSJc7NKv49KxXi56hGnOcB70qHXP4O6LFj5wlTZmYl560E\nzcxKzonezKzknOjNzErOid7MrOSc6M3MSs6J3sys5JzoraVJWpD+ubGkTzX42t/u9vofjby+WbM4\n0Vur65wIMgL4dF8+mK5O2ZMuS+lGxC59ub5ZUTjRW1mcDOyazrqcKGlQutnFbelmF4cDSOqQdKOk\ny4GZ6bE/Sboj3cBhfHrsZGDl9Hq/TY91/vag9NozJN0r6ZMV154m6Y/phhAXdAYn6WQlm5ncI+mU\npv7LWNvrrUdj1iqOA74eEfsApIn9pYgYo2SHqJskTU3P3ZZkE4zOjSW+GBEvSloZuE3SJRHxLUlH\nRsS2FW0/CJSiAAABkUlEQVR0/vbwceC9JEs4rA3cLumG9L1tSDbTeRr4h6RdSNbP2T8iNk9jWz2D\nv79ZTe7RW1l0X072o8DnJE0HbgHWBEal791WkeQBJkq6G7iZZCXVTXtpayxwUSTmAdcD7yf5Irgt\nIv6dLhl8N7Ax8BKwSNIvJR0AvNbvv6VZPzjRW5kdFRHbpo+RFRuILOw8QVIHsBuwY0RsA0wHhvZy\n3WD5L5bO3v7rFcfeBFaIiDdJdlW7BNgbuKo/fxmz/nKit7KYD6xW8fpq4IjOG66SNquxm9XqwIsR\nsUjS5iR7A3daXOOG7Y3AQel9gLWBDwC3UWOTCkmrAMMj4krgP0jKPmZN4xq9tbrOnvQ9wJtpCeZ8\n4AySTSTuSteVnwcckJ5fuWTrVcBXJM0CHiQp33Q6F7hX0p0R8dnOz0XE/0raKW0zgG9ExDwlG8d3\nXw42SL6ALpc0lOTL4GsN+Zub1cnLFJuZlZxLN2ZmJedEb2ZWck70ZmYl50RvZlZyTvRmZiXnRG9m\nVnJO9GZmJedEb2ZWcv8HA/gNnv1yYeMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1090a8b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ada = AdalineSGD(epochs=15, eta=0.01)\n",
    "\n",
    "# shuffle data\n",
    "np.random.seed(123)\n",
    "idx = np.random.permutation(len(y))\n",
    "X_shuffled, y_shuffled =  X_std[idx], y[idx]\n",
    "\n",
    "# train and adaline and plot decision regions\n",
    "ada.train(X_shuffled, y_shuffled)\n",
    "plot_decision_regions(X_shuffled, y_shuffled, clf=ada)\n",
    "plt.title('Adaline - Gradient Descent')\n",
    "plt.xlabel('sepal length [standardized]')\n",
    "plt.ylabel('petal length [standardized]')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(1, len(ada.cost_)+1), ada.cost_, marker='o')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Sum-squared-error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What's Next?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although we covered many different topics during this article, we just scratched the surface of artificial neurons. \n",
    "\n",
    "In later articles, we will take a look at different approaches to dynamically adjust the learning rate, the concepts of \"One-vs-All\" and \"One-vs-One\" for multi-class classification, regularization to overcome overfitting by introducing additional information, dealing with nonlinear problems and multilayer neural networks, different activation functions for artificial neurons, and related concepts such as logistic regression and support vector machines.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='./images/perceptron_activation.png' alt=''>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[back to top](#Sections)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[1] F. Rosenblatt. The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory, 1957.\n",
    "\n",
    "[2] W. S. McCulloch and W. Pitts. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4):115–133, 1943.\n",
    "\n",
    "[3] B. Widrow et al. Adaptive ”Adaline” neuron using chemical ”memistors”. Number Technical Report 1553-2. Stanford Electron. Labs., Stanford, CA, October 1960.\n",
    "\n",
    "[4] B. Recht and C. R ́e. Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences. arXiv preprint arXiv:1202.4184, 2012.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
